Identifying discrepancies between events from disparate systems

ABSTRACT

A system and method for determining event discrepancies between disparate systems is described. The system can receive sets of events from multiple systems. The system can normalize the sets of events to convert them into normalized events. From a normalized set of events, the system can concurrently generate a plurality of event arrays. The system can apply one or more rules to the event arrays to generate at least one event array parameter.

The present application claims priority to U.S. Provisional PatentApplication No. 62/553,675, filed Sep. 1, 2017, entitled PHARMACY ITEMEQUIVALENCE, U.S. Provisional Patent Application No. 62/554,145, filedSep. 5, 2017, entitled RELATING PHARMACY ITEM DATA FROM DIFFERENTSYSTEMS, and U.S. Provisional Patent Application No. 62/553,641, filedSep. 1, 2017, entitled OBFUSCATING PERSONAL HEALTH INFORMATION, each ofwhich is incorporated herein by reference in its entirety.

BACKGROUND

Pharmaceutical items (for example, drugs, diluents, medical and surgicalsupplies, gauze, scissors, needles, labels, baggies, bandages,packaging, vial, syringes, and/or other items that the pharmacy isresponsible for) such as medications (for example, drugs, diluents,etc., in solid or liquid form) can be difficult to track in a healthcareenvironment. For example, within the healthcare environment, there canbe significant waste, loss, or abuse of medications. Furthermore, it canbe difficult to identify equivalent pharmacy items. Furthermore, it canbe difficult to communicate pharmacy item event data withoutcommunicating personal health information.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a block diagram illustrating an embodiment of a pharmacy binwith multiple pharmacy item containers placed therein, each pharmacyitem container having an RFID tag attached thereto.

FIG. 2A is a block diagram illustrating an embodiment of an RFID readingand processing system.

FIG. 2B is a flow diagram illustrative of an embodiment of a routineimplemented by a system for dynamically scanning RFID tags.

FIG. 2C is a schematic diagram illustrating an embodiment of thecontainer with a pharmacy bin placed therein.

FIG. 3 is a flow diagram illustrative of an embodiment of a routineimplemented by a system for determining equivalence between pharmacyitems.

FIG. 4 is a block diagram illustrating an embodiment of a verificationenvironment for associating information across multiple health caresystems.

FIG. 5 is a flow diagram illustrative of an embodiment of a routine fortracking pharmacy items.

FIG. 6 is a flow diagram illustrative of an embodiment of a routine toidentify matching events from different systems of an environment.

FIG. 7 is a flow diagram illustrative of an embodiment of a routine tomatch events from the systems of an environment.

FIG. 8 is a block diagram illustrating an embodiment of different datastructures that can be used to map relevant information between systems.

FIG. 9 is a data flow diagram illustrative of an embodiment ofcommunications between different systems to identify discrepancies inevents.

FIG. 10 is a flow diagram illustrative of an embodiment of a routine tocommunicate event data information between systems with personal healthinformation (PHI) obfuscated.

FIG. 11 is a block diagram illustrating an embodiment of generation ofexample events arrays.

FIG. 12 is a flow diagram illustrative of an embodiment of a routine todetermine event discrepancies between disparate systems.

FIG. 13 illustrates an example hospital floor map indicative of anexample clustering of providers into their respective care areas.

FIGS. 14A-14B illustrate example graphical user interfaces (GUIs)displaying various metrics associated with a risk of drug diversion,according to some embodiments.

DETAILED DESCRIPTION

Pharmacy Items and Bins

FIG. 1 is a block diagram illustrating an embodiment of a pharmacy bin100 with multiple pharmacy item containers 102 placed therein, eachpharmacy item container 102 having an RFID tag 104 attached thereto. Thebin 100 can be made of a material, such as plastic, that reduces thelikelihood of interfering or blocking radio signals emitted from anantenna. In addition, the bin 100 can have its own RFID tag 106 attachedthereto.

As will be described in greater detail below, the pharmacy itemcontainers 102 can come in any number of forms, such as, but not limitedto, vials, bottles, syringes, boxes, ampoules, IV bags, baggies, orother containers configured to store a medication. Further, eachpharmacy item container 102 can store or contain one more pharmacyitems, such as, but not limited to, tablets, cartridges, creams,crystals, powder, fluid, film, gel, etc. For example, a bottle ofIbuprofen can store multiple Ibuprofen tablets and a vial of Propofolcan store 20 mL of Propofol, etc.

In some cases, different pharmacy item containers 102 in the bin 100 canstore the same or different types of pharmacy items. For example, onepharmacy item container 102 can store Propofol and another can storeOxycodone. In certain cases, the pharmacy items containers 102 can storesimilar pharmacy items. For example, one pharmacy item container 102 canstore a generic version of a drug and another can store the brandedversion of the same drug or active ingredient. Similarly, differentpharmacy item containers 102 can store the same drug manufactured bydifferent companies, the same drug with different concentrations,different routes of administration, different dosage forms, and/ordifferent package types, etc.

The RFID tags 104 can be passive, active, or battery-assisted passive.An active tag can include an on-board battery and can periodicallytransmit its tag information. A battery-assisted passive (BAP) caninclude a small battery on board and can be activated when in thepresence of an RFID reader. A passive tag can be less expensive andsmaller because it may not include a battery; instead, the tag can usethe radio energy transmitted by the RFID reader to power the integratedcircuit and generate a response.

The RFID tags 104 may either be read-only, for example, havingfactory-assigned tag information, such as a serial number or other data,that is used as a key into a database, or may be read/write, whereobject-specific data, serial number, can be written into the tag 104.Field programmable tags may be write-once, read-multiple, or “blank”tags that may be written with an electronic product code by the user,etc.

The RFID tags 104 can include an integrated circuit for storing andprocessing information and modulating and demodulating radio-frequency(RF) signals, and an antenna for receiving an interrogation signal froman RFID reader and transmitting the tag information. The tag informationcan be stored in a non-volatile memory of the integrated circuit orother memory on the RFID tag. Further, the RFID tag can include eitherfixed or programmable logic for processing the transmitting the taginformation.

In use, an RFID reader transmits an encoded radio signal to interrogatethe tag 104. The RFID tag 104 receives the message and then respondswith the tag information. When the RFID tag 104 is a passive tag, theRFID tag 104 can use the radio signal received from the RFID reader toenergize the tag 104. The tag information may be a unique tag serialnumber or may be product-related information such as a stock number, lotor batch number, production date, or other specific information storedon the RFID tag 104. In cases where tags have individual serial numbers,the RFID system can discriminate among several tags that might be withinthe range of the RFID reader and read them simultaneously. The RFIDreader can communicate the tag information to an information processingsystem, which can further process the tag information. As will bedescribed in greater detail below, an information processing system canuse the tag information to identify a bin, the pharmacy items in thebin, determine equivalences, or update item locations, among otherthings.

FIG. 2A is a block diagram illustrating an embodiment of an RFID readingand processing system 200. The system 200 includes an informationprocessing system 210 in communication with an RFID reading station 220that includes an RFID reader 222 and a container 226 for placing apharmacy bin 100 or pharmacy item container 102.

The information processing system 210 can be implemented in a variety ofways and can include one or more computers, servers, virtual machines,or the like. In some embodiments, the information processing system 210can be included as part of the RFID reading station 220 or RFID reader222. In certain embodiments, the information processing system 210communicates with the RFID reading station 220 via one or more networks,such as a local or wide area network. In some cases, the informationprocessing system can be implemented using a local computing device,such as a processor or computer located in the same room as the RFIDreading station 220, or on a remote computing device, such as a serverthat communicates with the local computing device via a local or widearea network.

The local computing device can include a computer application thatreceives tag information from the RFID reader and communicates with aremote computing device via a wide area network, such as the Internet.The remote computing device can be made up of one or more computingdevices, such as one or more servers, data stores, or virtual machines,and can store bin information such as the types of bins used in apharmacy environment. The remote computing device can also store recordsthat include information regarding individual bins and pharmacy items.In some cases, the information processing system 210 can receiveinformation from multiple RFID reading stations 220 located in a varietyof locations.

Using the tag information received from an RFID reading station 220, theinformation processing system 210 can identify equivalent pharmacy itemsin a bin 100 or in the container 226 and update bin data, as will bedescribed in greater detail below.

In addition to storing the current or most recent information regardingthe bins 100, the information processing system 210 may also store avirtual history for each bin 100. Such a virtual history can include,for example, a record of each transaction involving the bin 100 sincethe time it was created. Such transactions may include, for example,scans, database queries, updates such as restocking or removal of items,and so on. The virtual history can be output in the form of a report inresponse to a user request. In addition, the virtual history can be usedto gather data or statistics that may be useful for planning futuretasks such as bin 100 updates, item restocking, and so on.

Although FIG. 2A shows the RFID reading station 220 and the informationprocessing system 210 as separate features, they are not required to bephysically or functionally separate. For instance, informationprocessing features can be included as part of the RFID reading station220, such as the RFID reader 222. In general, the physical andfunctional implementation of information processing system 210 can bepartitioned between various forms of hardware, software, firmware, etc.

With continued reference to the embodiment illustrated in FIG. 2A, theRFID reading station 220 can include the RFID reader 222 and thecontainer 226. The RFID reader 222 can include a controller, such as amicroprocessor or microcontroller, and an antenna 224. At least aportion of the antenna 224 can be located within the container 226 thatis designed to receive the bin 100 during a read operation.

The RFID reader 222 can control the antenna 224 to communicate with RFIDtags 104 associated with items of bin 100, as well as any RFID tag 106associated with the bin 100 itself. In addition, the RFID reader 222 canreceive and process communications received by the antenna 224 from thebin 100. Although RFID reader 222 is shown outside of container 226, itcould alternatively be included within the container 226. Moreover,although the antenna is shown as being a part of the RFID reader 222, itwill be understood that they could be separated into differentcomponents.

In a typical read operation, the RFID reader 222 controls the antenna224 to interrogate any RFID tags within the container 226. Specifically,the RFID reader 222 causes the antenna 224 to emit a radio signal withinthe container 226. In response to the interrogation, the RFID tagscommunicate information to RFID reader 222 via antenna 224. In somecases, such as where the RFID tag is a passive RFID tag, the radioenergy from the radio signal can further be used to energize the tag andenable it to emit a radio signal that includes the tag information. Incertain cases, such as where the RFID tag is an active RFID tag, theRFID tag can include its own battery but can respond to the RFID reader222 with its tag information. The tag information can include an ID orother information stored in memory on the RFID tag.

The communicated tag information can be associated with correspondinginformation stored in a database, such as national drug code (NDC)identifiers, lot numbers, and expiration dates for individual items, anda bin identifier for the bin 100 as a whole. The RFID reader 222 cancommunicate the received tag information to information processingsystem 210 for storage and/or further processing.

Dynamic RFID Tag Scanning

As described herein, the RFID reader 222 can control the antenna 224 tointerrogate RFID tags within the container 226. In a typical readoperation, however, prior to the interrogation, the RFID reader 222 isnot aware of the presence of the RFID tags within the container 226.Rather, the RFID reader 222 and/or information processing system 210 candetermine that a particular RFID tag is located within the container 226based on the tag information received after the tag RFID has beeninterrogated. Often, the RFID reader 222 can “search” for RFID tags bytransmitting an encoded radio signal for a predetermined period of time(for example, 2-4 seconds), and can determine which, if any, RFID tagsare located within the container 226 based on received tag information.

However, in some cases, the predetermined period of time over which theRFID reader 222 searches for RFID tags is not of a sufficient durationto interrogate or energize each of the RFID tags present within the bin100. As a non-limiting example, the bin 100 can include one or more RFIDtags that are at least partially blocked or occluded from theinterrogation (for example, by one or more items in the bin 100).Accordingly, the one or more RFID tags that are at least partiallyblocked or occluded may take longer (for example, longer than thepredetermined period of time over which the RFID searches for tags) toregister or receive the encoded radio signal from the RFID reader 222.As a result, although a particular RFID tag may be within a bin 100 orwithin a container 226, in some cases the RFID reader 222 and/or theinformation processing system 210 may not receive tag information fromthat particular RFID tag, and therefore may not determine that theparticular RFID tag is within the bin 100 or the container to 226.

To address these and other problems, the RFID reader 222 can beconfigured to interrogate the pharmacy bin 100 for an increased periodof time, which can be of a sufficient duration so as to acquireinformation from each RFID tag within the bin 100. In some embodiments,to improve the likelihood that each RFID tag within the bin isinterrogated, the period of time over which the RFID reader 222 searchesfor RFID tags can be increased. For example, in some cases, the periodof time can be increased to a longer predetermined period of time (forexample, 20 seconds).

Alternatively, the period of time over which the RFID reader 222searches for RFID tags can be dynamic rather than fixed orpredetermined. For example, rather than a fixed duration of time overwhich the RFID scanner 222 interrogates or searches for RFID tags, theRFID scanner 222 can continue to scan for at least a predetermined timeafter each time it registers a new RFID tag. If the predetermined timeperiod expires before a new RFID tag is registered, then the RFIDscanner 222 can end the scan. However, if a new RFID tag is registeredbefore the expiration of the predetermined time period, then the RFIDscanner 222 can reset the predetermined time period and can continue toscan for RFID tags.

In general, if the bin 100 includes a plurality of RFID tags, then, dueto a variety of factors (non-limiting examples: distance from the RFIDreader 222, type of RFID tag, etc.), the RFID reader 222 can receive taginformation from the various RFID tags at different time intervals.Furthermore, during the period over which the RFID reader 222 searchesfor RFID tags, the same tag may transmit tag information to the RFIDreader multiple times. Accordingly, if multiple RFID are within the bin100, when the RFID reader 222 interrogates the bin, the RFID reader canreceive a plurality of messages from the RFID tags at various times, andat least some of those messages can be duplicate messages. In somecases, the RFID system can discriminate amongst the tag information toidentify pharmacy items in the bin 100, and can identify and/or ignoreduplicate messages, such as those messages from an RFID tag that havebeen processed and determined to be within the bin 100. If a new RFIDtag is registered before the expiration of the predetermined timeperiod, then the RFID scanner 222 can continue to scan and can restart atimer corresponding to the predetermined time period. If the timecorresponding to the predetermined time period expires before a new RFIDtag is registered, then the RFID scanner 222 can end the scan. Usingthis dynamic method of scanning, the system can ensure that all or mostof the RFID tags within the bin 100 are given enough time to receive theradio signal received from the RFID reader and energize. This can be thecase even if one or more RFID tags are blocked or occluded by otheritems in the bin 100.

Example Dynamic RFID Tag Scanning Routine

FIG. 2B is a flow diagram illustrative of an embodiment of a routine 250implemented by a system (such as system 200 of FIG. 2A) for dynamicallyscanning RFID tags. One skilled in the relevant art will appreciate thatthe elements outlined for routine 250 can be implemented by one or morecomputing devices or components that are associated with the system 200,such as the information processing system 210 or the RFID reader 222.Accordingly, routine 250 has been logically associated as beinggenerally performed by the system 200. However, the followingillustrative embodiment should not be construed as limiting.Furthermore, it will be understood that the various blocks describedherein with reference to FIG. 2B can be implemented in a variety oforders. For example, the system 200 can implement some blocksconcurrently or change the order as desired. Furthermore, it will beunderstood that fewer, more, or different blocks can be used as part ofthe routine 250. For example, in some embodiments, one or more of blocks252, 254, 256, or 258 are not implemented. However, it will beunderstood that any of the blocks (and more or different blocks) can beimplemented as part of routine 250.

At block 252, the system 200 activates an antenna 224 to search for RFIDtags. For example, the system 200 can control or cause the antenna 224to emit a radio signal. In some cases, as described herein, the antenna224 can emit the radio signal within the container 226 to interrogateany RFID tags associated with items of the bin 100, as well as any RFIDtag 106 associated with the bin 100 itself.

At block 254, the system 200 initiates (or re-initiates) a timer. Insome cases, once initiated (or re-initiated), the timer can beconfigured to expire after a predetermined period of time has passedsince initiation (or re-initiation). The counter can be a countdowntimer, count up timer, or other timer. Further, the predetermined periodof time can be any of various periods of time such as but not limited toless than a second, a few seconds, 5 seconds, 10 seconds, 15 seconds, 20seconds, 30 seconds, etc. (+/−a few seconds). In some cases, thepredetermined period of time can be configurable or selectable by auser. Alternatively, the predetermined period of time can be fixed.

As described above, prior to interrogation of the RFID tags, the system200 is not aware of which, if any, RFID tags reside within the container226. Instead, the system 200 can utilize the radio signal emitted fromthe antenna 224 to cause an RFID tag to transmit its tag information tothe system 200. The system 200 can process the tag information toidentify a bin, the pharmacy items in the bin, determine equivalences,or update item locations, among other things.

In some cases, a particular RFID tag can respond more than once to theradio signal(s) emitted by the antenna 224. For example, as the antenna224 emits a radio signal over a period of time, the particular RFID tagmay receive the radio signal multiple times and, as a result, sends itsreply multiple times. In some cases, the system 200 can discriminateamongst the tag information to identify RFID tags that correspond to newor unprocessed RFID tags. For example, based on the tag information, thesystem 200 can identify and/or ignore duplicate messages, such as thosemessages (or tag information) that have already been processed. Further,in certain embodiments, the system 200 can instruct an RFID tag that hasalready responded to not respond further.

At block 256, the system 200 can determine whether unprocessed taginformation has been received prior to the expiration of the timer. As anon-limiting example, new tag information can include tag informationthat has not been previous processed by the system 200, for example,since the time of the initial activation of the antenna (block 252). Forexample, as described above, a system 200 can receive tag informationfrom an RFID tag multiple times. When the system 200 first receives thetag information, the system has not yet processed that tag information.Accordingly, that tag information can be identified as unprocessed, andthe system 200 can process that tag information to identify a bin, thepharmacy items in the bin, determine equivalences, or update itemlocations, among other things. In contrast, when the system 200 receivesthat same tag information again (for example, via a duplicate message),the system has already processed that tag information, as describedabove. Accordingly, the duplicate tag information can be identified asprocessed tag information.

Referring to block 256, if new tag information has been received priorto the expiration of the timer that was initiated or re-initiated atblock 254, then the system 200 can return to block 254 and re-initiatethe timer. Moreover, the system 200 can continue to cause the antenna224 to emit the radio signal such that it continues to search for RFIDtags. However, if no new tag information has been received prior to theexpiration of the timer, then the system 200 proceeds to block 258. Atblock 258, the system deactivates the antenna 224 such that it stopssearching for RFID tags. In this way, the system can dynamically adjustthe amount of time for which the antenna is activated. For example, ifno tag information is received after five seconds and the timer is setfor five seconds, then after ten seconds the system 200 can deactivatethe antenna. With continued reference to the example, if no taginformation is received after seven seconds, then the system 200 candeactivate the antenna after twelve seconds.

It will be understood that the various blocks described herein can beperformed in a different sequence, can be added, merged, or left outaltogether, and that the routine 250 can implement one or more of theblocks concurrently and/or change the order, as desired. For example,the routine 250 can concurrently activate the antenna (block 252) andinitiate a timer (block 254). Alternatively, the routine 250 caninitiate the timer (block 254) before activating the antenna (block252).

Furthermore, it will be understood that fewer, more, or different blockscan be used as part of the routine for dynamically scanning RFID tags.As a non-limiting example, in some embodiments, the predetermined periodof time can be adjusted (for example, extended or shortened) when or ifthe timer is re-initiated. For example, each time the system 200 returnsto block 254 and re-initiates the timer, the predetermined period oftime (e.g., the time that the system 200 will wait before proceeding toblock 258 to deactivate the antenna) can be reduced, such as by one ormore seconds. Alternatively, each time the system 200 returns to block254 and re-initiates the timer, the predetermined period of time can beincreased, such as by one or more seconds.

Example RFID Reading and Processing System

FIG. 2C is a schematic diagram illustrating an embodiment of thecontainer 226 with a pharmacy bin 100 placed therein. In the illustratedembodiment of FIG. 2B, the container 226 comprises an enclosed space forreceiving bin 100. The left side of FIG. 2B shows container 226 withdoors 228 opened to receive bin 100, and the right side of FIG. 2B showscontainer 226 with doors 228 closed to perform a read operation. The useof an enclosed space allows RFID tags to be read without interferencefrom objects in the surrounding environment, thereby avoiding falsepositives from RFID tags on items located in the pharmacy area. In somecases, container 226 can include electromagnetic shielding material,such as a metal, to provide electromagnetic shielding. For example, theentire container 226 can be made of electromagnetic shielding material.In addition or alternatively, the container 226 can include one or morelayers of electromagnetic shielding material and/or one or more layersof non-electromagnetic shielding, such as plastic.

In some embodiments, the RFID reading station 220 or container 226 isrestricted to receiving only one bin at a time. This restriction may beimposed in a variety of ways, for instance, by configuring an enclosureto accommodate only one bin or interrogating bin tags prior to scanningto ensure that no more than one bin tag is present. In certainembodiments, RFID reading station 220 may be specifically configured toallow concurrent scanning of multiple bins. For example, two bins can beplaced in RFID reading station 220 or container 226 and scannedconcurrently, or the RFID reading station 220 can include multiplecontainers 226 that include different pharmacy bins 100. In some cases,the system can analyze the data from the multiple bins separately.

Pharmacy Item Equivalence

FIG. 3 is a flow diagram illustrative of an embodiment of a routine 300implemented by a system (such as system 200 of FIG. 2A) for determiningequivalence between pharmacy items. One skilled in the relevant art willappreciate that the elements outlined for routine 300 can be implementedby one or more computing devices or components that are associated withthe system 200, such as the information processing system 210 or theRFID reader 222. In addition or alternatively, routine 300 can beimplemented by one or more systems, such as the dispensing system 404,patient information system 406, administration system 408, or billingsystem 410 of FIG. 4. Accordingly, routine 300 has been logicallyassociated as being generally performed by the system 200. However, thefollowing illustrative embodiment should not be construed as limiting.Furthermore, it will be understood that the various blocks describedherein with reference to FIG. 3 can be implemented in a variety oforders. For example, the system 200 can implement some blocksconcurrently or change the order as desired. Furthermore, it will beunderstood that fewer, more, or different blocks can be used as part ofthe routine 300. For example, in some embodiments, one or more of blocks302, 304, 306, 308, 310, 312, 314 are not implemented. However, it willbe understood that any of the blocks (and more or different blocks) canbe implemented as part of routine 300.

At block 302, the system 200 receives tag information from RFID tags. Asdescribed herein, tag information can include a unique tag serial numberor product-related information, such as a stock number, lot or batchnumber, production date, or other specific information stored on theRFID tag(s) 104 or 106. In some embodiments, the receipt of taginformation can be in response to an antenna 224 interrogating the RFIDtag(s) 104 or 106 by emitting radio signals within the container 226.For example, the system 200 can cause the antenna 224 to emit a radiosignal that activates the RFID tag(s) 104 or 106 and causes the RFIDtag(s) 104 to respond with the tag information, or an indicationthereof.

At block 304, the system 200 identifies a pharmacy bin 100 based atleast in part on the received tag information. For example, as describedherein, at least one RFID tag can be associated with the pharmacy bin100 (non-limiting example: FIG. 1 illustrates the RFID tag 106 attachedto the pharmacy bin 100). The system 200 can use the tag informationfrom the associated RFID tag (for example, the RFID tag 106) to identifythe pharmacy bin 100. For example, the tag information can include anidentifier that corresponds to bin data stored in a database(non-limiting examples of bin data: data relating to types of pharmacyitems that are to be included in the pharmacy bin 100, data relating toparameters for determining item equivalence, or data relating to a binhistory (for example, items that were in the bin when last interrogated,items that were previously in the bin and later removed, etc.)). In someembodiments, the tag information can include the bin data.

At block 306, the system 200 identifies the pharmacy items presentwithin the RFID tag reading station 220, within the container 226,and/or within the pharmacy bin 100. For example, as described herein,the tag information from the various RFID tags can include identifiersthat correspond to pharmacy item data stored in a database. The systemcan use the identifiers to lookup the pharmacy item data and identifywhich, if any, pharmacy items are within the RFID tag reading station220, within the container 226, and/or within the pharmacy bin 100.

The pharmacy item data can include various characteristics of thepharmacy item, as described herein, or can include other information,such as, but not limited to, the number of times a pharmacy item (or itsRFID tag) has been scanned, the previous location(s) of the pharmacyitem, users that have interacted with or used the pharmacy item or RFIDtag, and/or the bin(s) with which the pharmacy item has been, or is,associated, expiration date, etc.

At block 308, the system 200 identifies base equivalent pharmacy items.In some cases, the system can identify the base equivalent pharmacyitems by comparing one or more of various characteristics of thepharmacy items, such as but not limited to, product identifier, name(s),manufacturer, concentration, per-unit concentration, schedule, fillamount, package type, package size, route of administration, dosageform, etc.

The product identifier can refer to a unique identifier for the pharmacyitem, such as, but not limited to a universal product identifier (UPI),national drug code (NDC), etc. The name can refer to the brand orgeneric name of a pharmacy item, or an active ingredient of a drug. Themanufacturer can refer to the entity that manufactures the pharmacyitem. If the pharmacy item is a drug, the manufacturer can refer to thedrug manufacturer. The concentration can refer to the concentration ofthe pharmacy item, such as 20 mg/mL for a drug. The per-unitconcentration can refer to the concentration by unit. For example, adrug with a concentration of 20 mg/20 mL would have a per-unitconcentration of 1 mg/1 mL. The schedule for a pharmacy item can referto the FDA schedule for drugs (e.g., Schedule 1-5).

The package size can refer to the size of the package or capacity for apackage. For example, the package size for a vial could be expressed inthe amount of mL that the vial can hold and the package size for pillscould be expressed in the number pills that the container could hold,etc. The fill amount can refer to the amount of a pharmacy item in apharmacy item container. For example, if a package size is 30 mL, butonly 20 mL is in the pharmacy item container, then the fill amount canbe 20 mL.

The package type for a pharmacy item can refer to how the pharmacy itemis packaged, such as, but not limited to, ampule, atomizer, bag, blisterpack, bottle (for example, plastic, glass, pump, spray, etc.), dropper,box, can, cup, cartridge, case, container (for example, plastic, glass,pump, etc.), inhaler, jar, pack, package, packet, pen, pouch, roll,syringe (for example, plastic, glass, etc.), strip, tube, tray, vial(for example, dispensing, multi-dose, bulk, single-dose, single-use,etc.), etc.

The route of administration can refer to the method in which thepharmacy item is administered to a patient, including, but not limitedto, epcutaneous, epidural, gastrostomy, inhalational, intraarterial,intraarticular, intracardiac, intracerebral, intradermal, intramuscular,intraocular, intravesical, intravenous, nasal, oral, rectal,subcutaneous, sublingual and buccal, transdermal, transmucosal, etc.

The dosage form can refer to can refer to the form of the pharmacyitems, including, but not limited to, aerosol, balm, capsule, cream,crystals, dermal patch, drops, ear drops, eye drops, gel, inhaler,intradermal, intramuscular, intraosseous, intraperitoneal, intravenous,liquid solution, liquid suspension, lotion, nebulizer, ointment, pastes,pill, powder/talc, skin patch, subcutaneous, suppository, tablet,vaporizer, etc.

In some embodiments, the system can determine that pharmacy items arebase equivalent if the following characteristics match: route ofadministration, dosage form, and per-unit concentration, and/or if thefollowing characteristics are comparable or similar: name (e.g., genericor brand or active ingredient correspond to each other) and packagetype. For example, the system can identify some package types ascomparable (e.g., vial and ampule) and some package types are notcomparable (e.g., vial and tabs). In certain embodiments, the system 200can be size agnostic in determining base equivalence. For example, thesystem can determine that a vial of 10 mL of Propofol is base equivalentto a vial of 30 mL of Propofol.

At block 310, the system 200 obtains a use-case policy. In some cases,the system 200 can obtain the use-case policy based on the identifiedbin and/or based on results of the base equivalent pharmacy items. Forexample, the bin data can identify a specific use-case policy for thebin and/or the system can determine a use-case policy based on thesimilarity between the base equivalent pharmacy items. In certainembodiments, the system can prompt a user to enter or select theuse-case policy. The use-case policy can correspond to a particular usecase for the pharmacy items, such as but not limited to, storage in abin, pharmacy kit creation, etc.

At block 312, the system 200 identifies use-case equivalent pharmacyitems. In certain embodiments, the system uses the characteristics ofthe base equivalent pharmacy items and the use-case policy to identifyuse-case equivalent pharmacy items. In some case, the system can usemore characteristics to determine the use-case equivalence than wereused for the base equivalence. For example, in addition to having thesame per-unit concentration, to identify pharmacy items as use-caseequivalent, the system can require that the pharmacy items have the sameconcentration. In certain cases, the system can usepreservative/no-preservative to determine use-case equivalence, but notto determine base equivalence. As a non-limiting example, one use-casepolicy can indicate that to be equivalent, pharmacy items must have thesame concentration, package size, package type, and name. Anotheruse-case policy can indicate that to be equivalent, all characteristicsof the pharmacy items must be the same (e.g., same identifier,manufacturer, name, package size, package type,preservative/non-preservative, etc.). Accordingly, using the use-casepolicy, the system can identify pharmacy items that are use-caseequivalent. Use-case policies can also or alternatively use package sizevs. fill amount, package size, etc. to determine use-case equivalentpharmacy items.

At block 314, the system 200 causes a display to display the results. Insome cases, the results can include an identification of some or allbase equivalent pharmacy items that were scanned and/or use-caseequivalent pharmacy items for one or more use cases. Further, the system200 can identify the pharmacy items in the bin that were neither baseequivalent nor use-case equivalent.

It will be understood that the various blocks described herein can beperformed in a different sequence, can be added, merged, or left outaltogether, and that the routine 300 can implement one or more of theblocks concurrently and/or change the order, as desired. For example,the routine 300 can concurrently identify the bin and identify thepharmacy items. Alternatively, the routine 300 can identify the pharmacyitems (block 306) before identifying the bin (block 304). Similarly, theroutine 300 can obtain a use-case policy (block 310) prior toidentifying base equivalent pharmacy items (block 308), and routine 300can identify base equivalent pharmacy items (block 308) and use-caseequivalent pharmacy items (block 312) concurrently or in any order. Inaddition, the routine 300 can display results (block 314) at any pointthroughout the routine 300.

Furthermore, it will be understood that fewer, more, or different blockscan be used as part of the routine 300 to identify equivalent pharmacyitems (or use-case equivalent pharmacy items). For example, in someembodiments, blocks 312 and 314 can be omitted such that the routine 300identifies base equivalent pharmacy items without determining use-caseequivalent pharmacy items. In addition or alternatively, block 308 canbe omitted such that the routine 300 identifies use-case equivalentpharmacy items without identifying base equivalent pharmacy items.

Furthermore, in some embodiments, block 304 can be removed such that theinformation processing system does not identify the bin. In suchembodiments, the information processing system can identify equivalencebased at least in part on the pharmacy items found or identified withinthe RFID tag reading station. For example, a number of pharmacy itemscan be placed in the RFID tag reading station. After interrogation, theinformation processing system can identify which, if any, of thepharmacy items are equivalent to each other. In some cases, the routine300 can identify different groups of pharmacy items that are equivalent.For example, the information processing system can identify a singlegroup of equivalent pharmacy items and identify all other pharmacy itemsas not equivalent.

In certain embodiments, the routine 300 can determine different levelsof equivalence. For example, a first level of equivalence can be basedon an active ingredient in the drug, a second level of equivalence canbe based on the active ingredient and concentration, and a third levelcan be based on the active ingredient, concentration, and one or moreadditional parameters, such as package type, volume, route ofadministration, etc. In certain cases, the different levels ofequivalence can be used by the system to determine base equivalenceand/or use-case equivalence. For example, in some cases, the system maydetermine a base equivalent based on a single level of equivalence, suchas the first level of equivalence, while the system may determine theuse-case equivalence based on multiple levels of equivalence, such asthe first and second level, the second and third level, or the first andthird levels of equivalence. Alternatively, in some cases, the systemmay determine a base equivalent based on multiple levels of equivalenceand/or the use-case equivalence based on a single level of equivalence.

Health Care Data Across Different Systems

Medical service providers often use different systems located at variouscare sites to generate or monitor health care data (non-limitingexamples: data related to events that occur in relation to a system,including event parameters such as drug identifiers, action identifiers,timing identifiers, patient identifiers, location identifiers, or thelike). Being able to accurately associate (non-limiting examples: share,combine, match, relate, link, or the like) the events from the differentsystems can be important in effective health care management. However,as described herein, associating events can be complicated when patientsreceive care in multiple settings or when organizations and providersuse different systems to share records electronically. Even within asingle healthcare organization, the wide range of systems for clinicaland administrative services can create obstacles to associating theevents.

In general, a particular system can assign its own identifiers to eachpatient, provider, or pharmacy item whose data it maintains. However,among other things, a lack of uniformity in the management of dataacross the systems makes it challenging for a computing device toassociate the health care data across multiple systems. For example,while individuals may use subjective criteria to associate events, thesystem 200 can utilize objective criteria to associate events, and itcan be challenging to accurately associate events using objectivecriteria when the data is not uniformly managed. As a non-limitingexample, use of a first system to track a dispensing event (non-limitingexample: the dispensing of a medication) and a second, different systemto track an administration event (non-limiting example: theadministration of the medication) can make it difficult to associateevents, even though these events may be associated with the samepatient, same drug, same provider, etc. Furthermore, the differentsystems can increase the challenge in identifying waste, identifyingmissing medications, or identify duplicate items or entries, which canincrease the costs for the medical service providers or can lead tofraud or drug abuse, among other things.

To further complicate medication tracking, the different systems maystore different information about the patient or pharmacy item that mayoverlap but is not identical. Thus, for example, it can be difficult todetermine whether an administered medication or a discarded medicationcorresponds to a dispensed medication.

Efforts to address these and other difficulties that arise from the useof different systems to generate or monitor health care data are furthercomplicated by the need to protect patient privacy and security. Forexample, the communication of protected health information (PHI)(non-limiting examples: demographic information, medical histories, testand laboratory results, mental health conditions, insurance information,and other data that a healthcare professional collects to identify anindividual and determine appropriate care) is highly regulated. Thus, itcan be difficult to associate events or other data between the differentsystems without communicating PHI.

FIG. 4 is a block diagram illustrating an embodiment of a verificationenvironment 400 for associating information across multiple health caresystems. In some cases, the association is HIPAA-compliant. In theillustrated embodiment, the verification environment 400 includes averification system 402, a dispensing system 404, a patient informationsystem 406, an administration system 408, and a billing system 410, eachcommunicably connected via network 412.

The dispensing system 404 can be implemented in a variety of ways andcan include one or more computers, servers, virtual machines, or thelike. In certain embodiments, the dispensing system 404 communicateswith other systems (for example, the verification system 402, patientinformation system 406, administration system 408, or billing system410) via one or more networks 412, such as a local or wide area network.In some cases, the dispensing system 404 can be implemented using alocal computing device, such as a processor or computer, or on a remotecomputing device, such as a server that communicates with a localcomputing device via a local or wide area network.

The dispensing system 404 can be configured include one or moreprocessing devices or data stores for controlling or tracking thedispensing, wasting, returning, or transferring of drugs. In some cases,dispensing system 404 is configured to store, dispense, or receivemedications while controlling or tracking drug distribution. Examplenon-limiting embodiments of the dispensing system 404 include one ormore of various drug dispensing cabinets (non-limiting example: anautomated dispensing medication cabinet (ADCs)), carousels, kits, pickpickers, box pickers, or the like.

The dispensing system 404 can be configured to store drugs (such as, butnot limited to, tablets, cartridges, creams, crystals, powder, fluid,film, gel, etc.) for dispensing. For example, the dispensing system 404can include a computerized drug storage device or cabinet and can beconfigured to dispense precise amounts of a particular drug ordetermined drug equivalent. In some cases, the dispensing system 404 candispense a drug in response to a request from a user. For example, auser can request to receive a particular amount of a drug from a drugdispensing cabinet or other dispensing location, and the drug dispensingcabinet or other dispensing location can dispense the drug.

The dispensing system 404 can identify, track, or communicate dispensingdata. For example, the dispensing system 404 can identify, track, orcommunicate dispensing event parameters such as, but not limited to, adrug identifier of the dispensed drug (non-limiting examples: drug name,drug quantity, drug concentration, package size, or other drugidentifying information), an action identifier (non-limiting example: inindication of the dispensing event), a timing identifier (non-limitingexamples: a time or date associated with the dispensing event), apatient (or the intended recipient of the drug) identifier (non-limitingexamples: patient name, birth date, height, weight, social securitynumber, or other patient information), provider identifier (non-limitingexamples: nurse, doctor, or provider name, or other provider informationindicative of the user(s) that requested or received the drug), or alocation identifier (non-limiting examples: a care area associated withthe provider or the patient, a location of the dispensing event, etc.).

In a typical scenario, the amount of drug dispensed from the dispensingsystem 404 is the same amount of the drug that is administered to thepatient. However, in some circumstances (non-limiting examples: the userrequested too much of the drug to be dispensed, the dispensing systemdispensed too much of the drug, the patient's requirements changed, orthe like), the amount of the drug that is dispensed is not equal to theamount of the drug that is administered. In other words, there is aportion of the drug that remains or is leftover after the drug wasadministered to the patient. In circumstances such as these, thisremaining portion of the drug should be properly disposed of orreturned.

As such, in some implementations, the dispensing system 404 (or anothersystem such as a waste system or a return system) can be configured toreceive drugs (such as, but not limited to, tablets, cartridges, creams,crystals, powder, fluid, film, gel, etc.) for discarding or returning.For example, the dispensing system 404 can include a computerized drugstorage device or cabinet and can be configured to receive and trackprecise amounts of a particular drug to be disposed of or returned forre-use. In some cases, the dispensing system 404 can receive a drug inresponse to a request from a user. For example, a user can request todispose of or return a particular amount of a drug into a bin or otherlocation, and the bin can receive and/or store the drug.

In some cases, the dispensing system 404 can identify, track, orcommunicate waste and/or return event data. For example, the dispensingsystem 404 can identify, track, or communicate waste event parametersand/or return event parameters such as, but not limited to: a drugidentifier of the wasted/returned drug (non-limiting examples: drugname, drug quantity, drug concentration, package size, or other drugidentifying information), an action identifier (non-limiting example: inindication of the waste/return event), a timing identifier (non-limitingexamples: a time or date associated with the waste/return event), apatient (or the intended recipient of the drug) identifier (non-limitingexamples: patient name, birth date, height, weight, social securitynumber, or other patient information), provider identifier (non-limitingexamples: nurse, doctor, or provider name, or other provider informationindicative of the user(s) that returned/wasted the drug), or a locationidentifier (non-limiting examples: a care area associated with theprovider or the patient, a location of the waste/return event, etc.). Insome cases, to return or dispose of a drug, a provider may need someoneto witness him/her performing the wasting or returning. Accordingly, insome cases, the waste and/or return event parameters can include awitness identifier. In some cases, the witness identifier can beincluded in or with the provider identifier.

In some cases, the dispensing system 404 can collect or trackinformation associated with pharmacy items in a pharmacy kit that ischecked out from a pharmacy. Similarly, the dispensing system 404 cancollect or track information associated with a drug when the drug isreturned to a pharmacy or cabinet, wasted, or transferred betweenpersons, etc.

Additional or alternative non-limiting examples of dispensing ortracking systems are described in U.S. Pub. No. 2015/0142469, filed Jan.23, 2015, entitled “Management of Pharmacy Kits,” and U.S. Pub. No.2017/0212993, filed Feb. 7, 2017, entitled “Medication Tracking,” eachof which is hereby incorporated herein by reference in its entirety.Furthermore, although only one dispensing system 404 is shown in FIG. 4,it will be understood that the environment 400 can include multipledispensing system. Moreover, in some cases, the dispensing system 404can be separated into multiple distinct systems, such as a wastemanagement system configured to store and track wasted or disposeddrugs, a return management system configured to store and track returneddrugs, and/or a different system for track and dispensing drugs.

The administration system 408 can be implemented in a variety of waysand can include one or more computers, servers, virtual machines, or thelike. In certain embodiments, the administration system 408 communicateswith other systems (for example, the verification system 402, dispensingsystem 404, patient information system 406, or billing system 410) viaone or more networks 412, such as a local or wide area network. In somecases, the administration system 408 can be implemented using a localcomputing device, such as a processor or computer, or on a remotecomputing device, such as a server that communicates with a localcomputing device via a local or wide area network.

The administration system 408 can include one or more processing devicesor data stores for controlling or tracking the administration of drugsto a patient. For example, as described herein, one or more drugs can bedispensed from a dispensing system, and at least some of the dispenseddrug can be administered to a patient. The administration system 408 canidentify, track, or communicate administration event data. For example,the administration system 408 can identify, track, or communicateadministration event parameters such as, but not limited to: a drugidentifier of the administered drug (non-limiting examples: drug name,drug quantity, drug concentration, package size, or other drugidentifying information), an action identifier (non-limiting example: inindication of the administration event), a timing identifier(non-limiting examples: a time or date associated with theadministration event), a patient (or the recipient of the drug)identifier (non-limiting examples: patient name, birth date, height,weight, social security number, or other patient information), provideridentifier (non-limiting examples: nurse, doctor, or provider name, orother provider information indicative of the user(s) that administeredthe drug), or a location identifier (non-limiting examples: a care areaassociated with the provider or the patient, a location of theadministration event, etc.).

In some cases, the administration system 408 can automatically track oridentify the amount of the drug that was administered to the patient. Inaddition or alternatively, the administration system 408 can receive aninput (for example, from the user that administered the drug or from auser that watched the administration) that identifies the drug, amountadministered, the user that administered the drug, or the like.

In some implementations, the one or more processing devices or datastores of the administration system 408 can be configured to storeelectronic medical records (EMRs). One non-limiting example of an EMRsystem is described in U.S. Pat. No. 5,924,074, incorporated herein byreference. In addition or alternatively, the one or more processingdevices or data stores of the administration system 408 can beconfigured for storing information related to admissions, discharges,transfers, etc. of patients. For example, the administration system 408can be configured to store information or identifiers corresponding toone or more patients such as, but not limited to, name, birth date,height, weight, medical history, health issues, allergies,prescriptions, treatments, drug administrations, social security number,admissions, discharges, transfers, etc.

The patient information system 406 can be implemented in a variety ofways and can include one or more computers, servers, virtual machines,or the like. In certain embodiments, the patient information system 406communicates with other systems (for example, the verification system402, dispensing system 404, administration system 408, or billing system410) via one or more networks 412, such as a local or wide area network.In some cases, the patient information system 406 can be implementedusing a local computing device, such as a processor or computer, or on aremote computing device, such as a server that communicates with a localcomputing device via a local or wide area network.

The patient information system 406 can include one or more processingdevices or data stores for storing or tracking patient information. Forexample, the patient information system 406 can be configured to storeinformation related to admissions, discharges, transfers, etc. ofpatients such as, but not limited to, patient name, birth date, height,weight, medical history, health issues, allergies, prescriptions,treatments, drug administrations, social security number, admissions,discharges, transfers, etc. In addition or alternatively, the patientinformation system 406 can be configured to store EMRs. Examplenon-limiting embodiments of an EMR system are described in U.S. Pat. No.5,924,074, filed Sep. 27, 1996, entitled “Electronic Medical RecordsSystem,” which is hereby incorporated herein by reference in itsentirety.

The billing system 410 can be implemented in a variety of ways and caninclude one or more computers, servers, virtual machines, or the like.In certain embodiments, the billing system 410 communicates with othersystems (for example, the verification system 402, dispensing system404, patient information system 406, or administration system 408) viaone or more networks 412, such as a local or wide area network. In somecases, the billing system 410 can be implemented using a local computingdevice, such as a processor or computer, or on a remote computingdevice, such as a server that communicates with a local computing devicevia a local or wide area network.

The billing system 410 can include one or more processing devices ordata stores for storing information related to the billing for the useof the medications. For example, the billing system 410 can beconfigured to store patient information related to admissions,discharges, transfers, or drug administration, such as, but not limitedto, patient name or other identifier, prescriptions, treatments, drugadministrations, admissions, discharges, or transfers.

The verification system 402 can include one or more processing devicesor data stores and can be configured to associate information (such asdate related to events) between or from one or more of the dispensingsystem 404, the patient information system 406, the administrationsystem 408, and/or the billing system 410. For example, the verificationsystem 402 can be communicatively coupled with one or more of thedispensing, patient information, administration, or billing systems404-410 via one or more networks 412. In some implementations, the oneor more networks 412 correspond to a local area network (LAN) or widearea network (WAN).

The verification system 402 can be configured to share, combine,aggregate, match, relate, or link data associated with one or more ofthe different systems 404, 406, 408, or 410. For example, as describedherein, each of the systems 402, 404, 406, 408, or 410 can becommunicatively coupled via a network 412, and the verification system402 can receive events or other data corresponding to one or more of thesystems 404, 406, 408, or 410 via the network 412. In addition oralternatively, in certain embodiments, the verification system 402 canreceive data associated with one or more systems 404, 406, 408, or 410via an intermediary, such as a portable memory device, CD, etc.

The verification system 402 can be configured to associate (non-limitingexamples: share, combine, aggregate, match, relate, or link) any of thevarious systems' data as described herein. For example, the verificationsystem 402 can be configured to associate one or more sets of events(non-limiting examples: dispensing events, administrating events, wasteevents, return events, billing events) from the various systems. In somecases, each event associated with a system can include plurality ofevent parameters including, but not limited to, a drug identifier, anaction identifier, a timing identifier, a patient identifier, a provideridentifier or a location identifier. In addition or alternatively,verification system 402 can be configured to associate EMRs, patientidentifying information, user identifying information (for example, theuser that dispense, administered, or discarded a drug), or the like.

As described herein, for example with respect to FIGS. 1-2B, in somecases, the verification system 402 can be configured to identifyequivalent drugs between one or more of the dispensing system 404, thepatient information system 406, the administration system 408, or thebilling system 410. Furthermore, as described herein, for example withrespect to FIG. 9, in some cases, the verification system 402 can beused to identify pharmacy item discrepancies between one or more of thedispensing system 404, the patient information system 406, theadministration system 408, or the billing system 410.

It will be understood that the verification system 402 can beimplemented in a variety of ways. For example, the verification system402 can include, but is not limited to, one or more processors,computers, servers, virtual machines, or the like. In addition oralternatively, the verification system 402 can include one or more of alocal computing device (such as a processor or computer located in thesame room as the dispensing system 404 and/or the administration system408) or on a remote computing device (such as a server that communicateswith the dispensing system 404 and/or the administration system 408 viaa local or wide area network). In some embodiments, the verificationsystem 402 can be included as part of the dispensing system 404, patientinformation system 406, administration system 408, or billing system410.

Although only one system of each type is shown in FIG. 4, it will beunderstood that the environment 400 can include multiple systems of eachtype, such as multiple dispensing systems 404, patient informationsystems 406, administration systems 408, or billing systems 410, each ofwhich can be communicatively coupled to each other and/or with theverification system 402 via one or more networks 412.

Pharmacy Item Tracking

FIG. 5 is a flow diagram illustrative of an embodiment of a routine 500for tracking pharmacy items. One skilled in the relevant art willappreciate that the elements outlined for routine 500 can be implementedby a system, such as the RFID reading and processing system 200,verification system 402, dispensing system 404, patient informationsystem 406, administration system 408, or billing system 410, or one ormore computing devices or components or a system, such as theinformation processing system 210 and/or the RFID reader 222. Forsimplicity, the routine 500 has been logically associated as beinggenerally performed by the system 200. However, the followingillustrative embodiment should not be construed as limiting.Furthermore, it will be understood that the various blocks describedherein with reference to FIG. 5 can be implemented in a variety oforders. For example, the system 200 can implement some blocksconcurrently or change the order as desired. Furthermore, it will beunderstood that fewer, more, or different blocks can be used as part ofthe routine 500. For example, in some embodiments, one or more of blocks502, 504, or 506 are not implemented. However, it will be understoodthat any of the blocks (and more or different blocks) can be implementedas part of routine 500.

At block 502, the system 200 receives tag information from RFID tags. Insome embodiments the receipt of tag information can be in response tothe antenna 224 emitting radio signals within the container 226, asdescribed in greater detail herein. For example, the system 200 cancause the antenna 224 to emit a radio signal that activates the RFIDtags and causes them to respond with the tag information.

At block 504, the system 200 determines that at least one pharmacy itemis associated with a different group of pharmacy items and/or in adifferent bin. As described herein, in some cases, the tag informationfrom the various RFID tags can include IDs that correspond to pharmacyitem data stored in a database. The system can use the IDs to lookup thepharmacy item data and identify the pharmacy items that are present inthe bin. In certain embodiments, the tag information can include thepharmacy item data.

In some cases, as described above, the pharmacy item data can includehistorical information regarding the corresponding pharmacy item. Forexample, the pharmacy item data can indicate the number of times thepharmacy item (or its RFID tag) has been scanned, the previous locationsof the pharmacy item, and/or the bin(s) with which the pharmacy item hasbeen, or is, associated. The system can use the historical informationto determine whether a pharmacy item is associated with a differentlocation and/or a different bin.

At block 506, the system 200 can update the pharmacy item data so thatthe pharmacy item is associated with a current location or bin. Forexample, if the pharmacy item data indicates that the pharmacy item isassociated with an RFID reading station at a different location, thesystem 200 can update the pharmacy item data to indicate that thepharmacy item is no longer located at the other location and/or that thepharmacy item is located at a new location. Similarly, if the pharmacyitem data indicates that the pharmacy item is associated with adifferent bin, the system 200 can update the pharmacy item data toindicate that the pharmacy item is no longer associate or located in thedifferent bin and/or that the pharmacy item is located in or associatedwith a new bin.

It will be understood that the various blocks described herein withrespect to routine 500 can be performed in a different sequence, can beadded, merged, or left out altogether, and that the routine 500 canimplement one or more of the blocks concurrently and/or change theorder, as desired. Furthermore, it will be understood that fewer, more,or different blocks can be used as part of the routine 500 to trackpharmacy items. For example, in some embodiments, block 506 can beomitted such that the routine 500 identifies at least one RFID tagassociated with another group, but does not update data of the anothergroup. Rather, the routine 500 can cause a display to display anindication to move the identified RFID tag to the bin or group to whichis associated.

Pharmacy Item Tracking Between Systems

FIG. 6 is a flow diagram illustrative of an embodiment of a routine 600to identify matching events from different systems of an environment,such as environment 400. One skilled in the relevant art will appreciatethat the elements outlined for routine 600 can be implemented using anyone or any combination of the systems in the environment 400, such asthe verification system 402, dispensing system 404, patient informationsystem 406, administration system 408, or billing system 410. Forsimplicity, the routine 600 has been logically associated as beinggenerally performed by the verification system 402. However, thefollowing illustrative embodiment should not be construed as limiting.Furthermore, it will be understood that the various blocks describedherein with reference to FIG. 6 can be implemented in a variety oforders. For example, the system 406 can implement some blocksconcurrently or change the order as desired. Furthermore, it will beunderstood that fewer, more, or different blocks can be used as part ofthe routine 600. For example, in some embodiments, one or more of blocks602, 604, 606, 608, or 610 are not implemented. However, it will beunderstood that any of the blocks (and more or different blocks) can beimplemented as part of routine 600.

At block 602, the verification system 402 receives sets of events fromone or more systems, such as the dispensing system 404, patientinformation system 406, administration system 408, and/or billing system410. Each event of the set of events can include a plurality of eventparameters, such as a drug identifier, an action identifier, a timingidentifier, a patient identifier, a provider identifier or a locationidentifier, as described herein. The events can correspond to dataentries in the different systems, such as drug dispensing oradministration entries, an electronic medical record entry, billinginformation for a drug, or the like.

At block 604, the verification system 402 identifies relevant portionsof the events to be used for mapping between systems. For example, insome cases, the relevant portions can include one or more eventparameters, such as a drug identifier, an action identifier, a timingidentifier, a patient identifier, a provider identifier or a locationidentifier, or a combination thereof. However, it will be understoodthat other portions of the event can be used.

At block 606, the verification system 402 identifies additionalinformation corresponding to the relevant portions or event parameters.For example, if the event parameter is a drug name, the verificationsystem 402 can identify other names for the drug (for example, genericor brand name(s), active ingredient, etc.), substitutes for the drug,medical descriptions of the drug (for example, description of drug in adictionary encyclopedia, drug reference document, etc.). As anotherexample, if the relevant portion is a provider name, the verificationsystem 402 can identify a care area (non-limiting examples: ER, OR, ICU,etc.) associated with the provider.

At block 608, the verification system 402 assigns a score or numericvalue to the identified relevant portions and/or the additionalinformation. For example, in some cases, the score assigned to therelevant portions and/or the additional information can be based atleast in part on the relevant portion's (or the additionalinformation's) similarity to the terms found in a training ornormalization database. For example, when a relevant portion is similarto or matches a word in the normalization database, that relevantportion can be assigned a relatively higher or better score. Similarly,when the additional information is similar to or matches a word in thenormalization database, that additional information can be assigned arelatively higher or better score. In contrast, relevant portions and/orthe additional information that are dissimilar to the words found in thenormalization database can be assigned lower values.

As a non-limiting example, and continuing with the example above, if therelevant portion is a drug name, the verification system 402 can assigna score to the name of the drug identified in the relevant portion basedon its comparison to the normalization database. Furthermore, theverification system 402 can assign a score to the additional information(e.g., other names for the drug), and/or to a description of the drugfound in the event. In some cases, the verification system 402 canassign a score to each separate word in the description or to multiplewords in the description (for example, not assign a score to “the”“or”), such as medically related terms in the description.

At block 610, the verification system 402 identifies matching events orevent parameters based on the scores assigned to the relevant portionsand/or the additional information. The matched events or eventparameters can correspond to event parameters identified in anormalization database, normalized event parameters, or events fromother systems. For example, using the score assigned to the relevantportions and/or the additional information, the verification system 402can build or generate an identifier for the relevant portions. In somecases, the identifier can include the score for the relevant portionsand the score for each piece of additional information that relates tothe relevant portions. The generated identifier for a relevant portionof an event from one system can be compared to the generated identifiersfor relevant portions and/or additional information of events from adifferent system or from a normalization database. The identifiers thatare most similar can be identified as a match. This process can berepeated for the relevant portions of each event from the system untilall of the relevant portions of the events from one system are mapped ormatched with the relevant portions of events from a different system, orare normalized with a normalized event parameter. For example, theprocess can continue until all drug names from one system are mapped toor match the drug names in a second system or are normalized or mappedto a normalized drug name (or event parameter).

It will be understood that the various blocks described herein withrespect to routine 600 can be performed in a different sequence, can beadded, merged, or left out altogether, and that the routine 600 canimplement one or more of the blocks concurrently and/or change theorder, as desired. For example, in some cases, the verification system402 can review the relevant portions and remove duplicates. In somecases, the events, score, relevant portions, additional information, orthe like can be stored in a data store for later retrieval. Furthermore,in some embodiments, block 606 can be omitted such that routine 600 doesnot identify additional information corresponding to the relevantportions. Accordingly, the routine 600 can assign a score to therelevant portions and can identify the related events based off theassigned score and/or the relevant portions.

As described herein, in some cases, to normalize an event, the systemcan map or match data received from a first system to normalized data ina normalization database. Accordingly, when a match is identified, thesystem can replace the event parameter with the normalized eventparameter or include the normalized event parameter as part of anormalized event.

Event Tracking Between Systems

FIG. 7 is a flow diagram illustrative of an embodiment of a routine 700to match events from the systems of an environment, such as environment400. One skilled in the relevant art will appreciate that the elementsoutlined for routine 700 can be implemented using any one or anycombination of the systems in the environment 400, such as theverification system 402, dispensing system 404, patient informationsystem 406, administration system 408, or billing system 410. Forsimplicity, the routine 700 has been logically associated as beinggenerally performed by the verification system 402. However, thefollowing illustrative embodiment should not be construed as limiting.Furthermore, it will be understood that the various blocks describedherein with reference to FIG. 7 can be implemented in a variety oforders. For example, the system 406 can implement some blocksconcurrently or change the order as desired. Furthermore, it will beunderstood that fewer, more, or different blocks can be used as part ofthe routine 700. For example, in some embodiments, one or more of blocks702-718 are not implemented. However, it will be understood that any ofthe blocks (and more or different blocks) can be implemented as part ofroutine 700.

At block 702, similar to block 602 of FIG. 6, the verification system402 receives one or more events from one or more systems. The events canbe received via a network 412 or via an alternate medium, such as aportable electronic storage device.

At block 704, the verification system 402 identifies one or more fieldsfor matching or normalizing the events. For example, each event caninclude a plurality of event parameters, such as a drug identifier, anaction identifier, a timing identifier, a patient identifier, a provideridentifier or a location identifier, and/or other data. Based on thesystems with data to be matched or the content of a normalized field,the system can determine which of the fields of the received event areto be retained and which are to be discarded. For example, in somecases, the system can remove the fields, event parameters, or other datathat is not to be used to match events or does not correspond to contentof a normalized event. In some cases, rather than removing data fromevents, the verification system 402 can disregard the information whenit matches events between the systems or normalizes the event.

At block 706, the verification system 402 places the identified portionsof the events from one system into a first array (for example, array 1of system 1 or A1S1). The identified portions of events from anothersystem can be placed in a second, different array (for example, A1S2).

In some cases, the different systems may have recorded the same event,and thus one or more of the arrays can have the same or similarinformation. Accordingly, at block 708, the verification system 402removes duplicate entries from the arrays. For example, the verificationsystem 402 can compare the first and second array to check forsimilarities. If the verification system 402 determines that the firstand second arrays satisfy a threshold similarity level, then the systemcan remove or ignore one of the entries as a duplicate. In some cases,the threshold similarity levels can include one or more matching orsubstantially similar event parameters. For example, if the first andsecond arrays each include entries having a same drug identifier,patient identifier, time identifier, and action identifier, then theverification system 402 can classify the entries as duplicates and canignore or more one of the duplicate entries.

At block 710, for A1S1, the verification system 402 generates a secondarray (A2S1) that includes one or more synonyms or alternate names orspelling for one or more entries in the A1S1. For example, if A1S1includes an entry “ibuprofen,” the second array can include entries for“Advil,” “NeoProfen,” “Caldolor,” “Motrin IB,” “isobutylphenylpropionicacid,” etc. The verification system 402 can include an indication thatthese entries relate to the “ibuprofen” entry in A1S1. In certainembodiments, each entry in A2S1 is related to one entry in A1S1,however, an entry in A1S1 can be related to multiple entries in A2S1.

At block 712, for A1S1, the verification system 402 generates a thirdarray (A3S1) that includes additional information for one or moreentries in A1S1. With continued reference to the “ibuprofen” entryabove, A3S1 can include entries for “nonsteroidal,” “anti-inflammatory,”“treatment” “pain,” “fever,” etc. Similar to A2S1, each entry in A3S1can be related to one entry in A1S1, however, an entry in A1S1 can berelated to multiple entries in A3S1. The additional information cancorrespond to information obtained from additional resources, such asdictionaries, encyclopedias, drug reference documents, etc. In certaincases, when multiple resources are used, A3S1 can include duplicates ofsome entries. For example, if two resources include the term“nonsteroidal,” “anti-inflammatory,” for “ibuprofen,” A3S1 can includean entry for each resource that includes that term.

At block 714, for each of A1S1, A2S1, and A3S1, the verification system402 generates a scoring array (e.g., B1S1, B2S1, B3S1) that includes ascore for each entry in the respective array. With continued referenceto the “ibuprofen” example, the scoring arrays can include a score for“ibuprofen,” “Advil,” “NeoProfen,” “Caldolor,” “Motrin IB,”“isobutylphenylpropionic acid,” “nonsteroidal,” “anti-inflammatory,”“treatment” “pain,” “fever,” etc. In some cases, the scores can be basedon the similarity of the entry with an entry in a normalizationdatabase. In some embodiments, the normalization database includes alist of drugs. However, for embodiments in which other content from themessages corresponds to the relevant content used for mapping, thenormalization database can include additional or different information.

At block 716, for each entry in A1S1, the verification system 402determines a vector based on the scores in the entries of B1S1, B2S1,B3S1 that are associated with the particular entry in A1S1. Withcontinued reference to the “ibuprofen” example, the verification system402 can generate a vector based on the score assigned to “ibuprofen,”“Advil,” “NeoProfen,” “Caldolor,” “Motrin IB,” “isobutylphenylpropionicacid,” “nonsteroidal,” “anti-inflammatory,” “treatment” “pain,” “fever,”a score assigned to a subset thereof. The vector can include the scorefor each entry and/or a combination of the scores, such as an average, acosine similarity, etc.

At block 718, the verification system 402 identifies a match betweenentries in the arrays A1S1 that correspond to system 1 with entries inthe arrays A1S2 that correspond to system 2 based on the generatedvectors. In some cases, to identify the match, the system compares thevector of one entry in the array A1S1 with the vector of each entry inthe array A1S2. In some cases, the entries with vectors that are themost similar are considered a match. In certain embodiments, theverification system 402 uses a cosine similarity function to identifymatches. For example, the cosine similarity between two vectors can be ameasure that calculates the cosine of the angle between them.Accordingly, cosine similarity can generate a metric that identifies howrelated two vectors are by looking at the angle of the vectors. Withcontinued reference to the “ibuprofen” example, system 2 may have anentry named “I-Prin,” and the arrays related to that entry may include“ibuprofen,” “Advil,” “NeoProfen,” “Caldolor,” “Motrin IB,”“isobutylphenylpropionic acid,” “nonsteroidal,” “anti-inflammatory,”“treatment” “pain,” “fever.” Further, based on the similar terms in thearrays, the vector for the “I-Prin” can be similar to the vector for“ibuprofen.” Based on the similarity, the verification system 402 candetermine that “ibuprofen” in system 1 corresponds to “I-Prin” in system2.

It will be understood that the various blocks described herein withrespect to routine 700 can be performed in a different sequence, can beadded, merged, or left out altogether, and that the routine 700 canimplement one or more of the blocks concurrently and/or change theorder, as desired. For example, rather than generating additional arrays(A2S1, A2S3), the verification system 402 can edit the array A1S1 or usesome other data structure for storing and comparing the relevantinformation. Furthermore, in some cases, the messages, score, relevantportions, additional information, or the like can be stored in a datastore for later retrieval.

It will be understood that fewer, more, or different blocks describedherein with respect to routine 700 can be utilized to normalize events.For example, as mentioned above, in some cases, the verification system402 can generate one or more vectors based on a similarity between anevent parameter and an entry in the normalization database. Based on thevector value(s), the verification system 402 can identify an entry inthe normalization database to which the event parameter corresponds(similar to matching entries in the arrays A1S1 that correspond tosystem 1 with entries in the arrays A1S2 that correspond to system 2 ofblock 718). Based on the match, the verification system 402 can replacethe event parameter in the received event with the normalized eventparameter or include a normalized event parameter corresponding to theevent parameter from the received event in a normalized event. In someembodiments, the verification system 402 can determine a normalizedevent parameter for each event parameter of the received event until theevent parameters are normalized, replaced with a normalized eventparameter, or a complete normalized event is generated. It will beunderstood that in some cases, the event parameter of the received eventmay be the same as the normalized event parameter.

Moreover, in certain embodiments, once events from different systems arenormalized, the verification system 402 can identify matches bycomparing normalized event parameters of normalized events associatedwith different systems. In certain embodiments, the normalized eventsfrom different systems can be matched without the use of a cosinesimilarity or vector. For example, in certain cases, the normalizedevent parameters can be matched based on identical matches between thedifferent normalized events. In this way, the normalized events canreduce the computing resources used to match disparate events betweensystems.

Mapping Information Between Systems

FIG. 8 is a block diagram illustrating an embodiment of different datastructures that can be used to map relevant information between systems.With reference to FIG. 8, messages 802 a and 802 b can include variouspieces of data related to an activity that involved Diprivan andHydromorphone, respectively. From the messages 802 a, 802 b, theverification system 402 can generate the array 804 that includes therelevant field from each message, which in this case is the drug name.

Using the first entry of the array 804, the verification system 402 cangenerate the array 806 that includes alternate names or synonyms forDiprivan and the array 808 that includes additional information that isobtained about Diprivan from one or more sources. For example, theadditional information can include other names for the drug (forexample, generic or brand name(s), active ingredient, etc.), substitutesfor the drug, medical descriptions of the drug (for example, descriptionof drug in a dictionary encyclopedia, drug reference document, etc.).From the arrays 804, 806, 808, an additional array 810 can be generatedthat includes the first entry from array 804, the entries from each ofthe arrays 806, 808, and a score associated with one or more of theentries. The scores assigned to the entries can be used to compareDiprivan in system A with other drugs in other systems or thenormalization database, as described in greater detail herein. Forexample, if another system refers to the same drug as Dipravan orPropofol, the verification system 402 can compare the scores or vectorsfrom array 810 with a score or vector corresponding to Dipravan orPropofol from the other system to identify a match. Similarly, theverification system 402 can use the vectors or data to normalize theevent parameter from system A.

Using the second entry of the array 804, the verification system 402 cangenerate the array 812 that includes alternate names or synonyms forHydromorphone and the array 814 that includes additional informationthat is obtained about Hydromorphone from one or more sources. From thearrays 804, 812, 814, an additional array 816 can be generated thatincludes the second entry from the array 804, the entries from each ofthe arrays 812, 814, and a score associated with the entries. The scoresassigned to the entries can be used to compare Hydromorphone in system Awith other drugs in other systems or the normalization database, asdescribed in greater detail herein.

Although illustrated for two messages, it will be understood that theverification system 402 can perform a similar array generation andcomparison for more messages from one or more systems. Furthermore, insome cases, the verification system 402 can perform additional steps.For example, the verification system 402 can remove duplicates from thearray 804 by comparing entries of the array 804 to determine if anymatches exist. If a match exists, the verification system 402 can removeone or the entries of the array 804 corresponding to the match. Inaddition, although illustrated as various arrays 804, 806, 808, 810,812, 814, 816, it will be understood that the verification system 402can use fewer or more arrays for each entry. For example, allinformation can be included in a single array, arrays 806 and 812,arrays 808 and 814, and arrays 810 and 816 can be combined into theirown arrays, respectively, or each of the arrays 806, 808, 810, 812, 814,816 can each correspond to its own array. Furthermore, althoughdescribed with respect to identifying related drugs, it will beunderstood that the verification system 402 can use the data structuresdescribed herein to identify other related information from differentsystems.

Discrepancies in Events Arrays

As described herein, medical service providers often use differentsystems located at various care sites to generate or monitor health caredata. In some cases, one or more systems (such as the verificationsystem 402) can be configured to associate events or other data from thedifferent systems. The associated events can be utilized to efficientlygroup, relate, and/or evaluate health care data such that discrepanciesor inconsistencies in the data can be identified and assessed, amongother things. By associating events from the different systems, theverification system 402 can reduce the processing resources used toprocess or analyze health care data.

FIG. 9 is a data flow diagram illustrative of an embodiment ofcommunications between different systems to identify discrepancies inevents. One skilled in the relevant art will appreciate that the dataflow outlined in FIG. 9 can be implemented using any one or anycombination of the systems in the environment 400, such as theverification system 402, dispensing system 404, patient informationsystem 406, administration system 408, or billing system 410. In thisexample, the data flow diagram includes communications between theverification system 402, the dispensing system 404, and theadministration system 408. However, the following illustrativeembodiment should not be construed as limiting.

At 1A, the verification system 402 requests data related to one or moredispensing events, which can include an instance in which the dispensingsystem 404 dispenses a drug or otherwise identifies that a drug wasdispensed. As described herein, the data related to a dispensing eventcan include one or more dispensing event parameters, which can include,but are not limited to, one or more of a drug identifier, an actionidentifier, a timing identifier, a patient identifier, a provideridentifier, a location identifier, or other information related to thedispensing event.

At 1A, the verification system 402 requests data related to one or moreadministration events, which can include an instance in which theadministration system 408 administers a drug or otherwise identifiesthat a drug was administered. As described herein, the data related toan administration event can include one or more administration eventparameters, which can include, but are not limited to, one or more of adrug identifier, an action identifier, a timing identifier, a patientidentifier, a provider identifier, a location identifier, or otherinformation related to the administration event.

It will be understood that the administration event data and/ordispensing event data can be collected using an RFID reading station(for example, RFID reading station 220) or via some other mechanism,such as a doctor or system user entering the information into thedispensing system 404 and/or administration system 408. Furthermore,dispensing events and administration events, as well as waste events andreturn events, can generally be referred to as events.

At 2A and 2B, the dispensing system 404 and the administration system408 process the request. At 3A and 3B, the dispensing system 404 and/orthe administration system 408 communicate the events to the verificationsystem 402.

At 4, the verification system 402 associates the events. In some cases,the dispensing events and the administration events can be in differentformats. Accordingly, as part of associating the data, the verificationsystem 402 can format the events into a common or normalized format. Forexample, the verification system 402 can format each event from thedifferent systems into an array and have an identifier assigned to eachfield in the array. In some cases, events received from the differentsystems can be referred to as pre-processed events and events that havebeen formatted or normalized can be referred to as processed events ornormalized events. For example, the identifiers can correspond to theevent parameters associated with each event, such as one or more of adrug identifier, an action identifier, a timing identifier, a patientidentifier, a provider identifier or a location identifier.

As part of normalizing the data, the system can associate parameters ofthe pre-processed event to a normalized parameter. For example, a drugidentifier in a pre-processed event may not match a normalized versionof that event parameter. As such, when generating the normalizedparameter, the system can use the normalized parameter in place of thecorresponding parameter from the pre-processed event.

In some embodiments, the verification system 402 can generate a vectorfor the parameters of a pre-processed event. The verification system 402can then use the vectors to identify a corresponding normalizedparameter. In some embodiments, any one or any combination of thefollowing can be used to generate a vector: brand name, generic name,concentration, package description, etc. Furthermore, in certainembodiments, the verification system 402 can normalize each eventparameter of the pre-processed events. In some cases, the verificationsystem 402 can concurrently generate normalized events. Accordingly,normalized events corresponding to events from different systems can begenerated concurrently. For example, in some cases, the verificationsystem 402 can utilize one or more processors to generate the normalizedevents in parallel, which can allow the system to efficiently identifydiscrepancies or potential abuse.

In some cases, to associate the data, the verification system 402 cancompare the contents of the different arrays to identify related events.In some cases, two or more events (or normalized events) are related ifthe two or more events have certain matching event parameters. Forexample, in some cases, events are related if the events have matchingpatient, drug, and/or location identifiers. In addition oralternatively, in some cases, events that have timing identifiers thatsatisfy a timing threshold can be related. As a non-limiting example, adispensing event can correspond to an instance in which morphine wasdispensed at 12:00 PM, to be administered to Patient A, and anadministration event can correspond to an instance in which morphine wasadministered at 12:14 PM to Patient A. In some cases, because thepatient (Patient A) and drug (morphine) match, and because the timeentries satisfy a timing threshold range of each other (for example, 1hour), the events can be identified as related events. It will beunderstood, however, that events can be related in various ways, and theprevious example should not be construed as limiting.

In some embodiments, the verification system 402 can generate a vectorfor each entry in an array. The verification system 402 can then comparethe vectors to identify events that are related, or as mentioned, tonormalize the events. In some embodiments, the verification system 402can use any one or any combination of the following to generate thevector: brand name, generic name, concentration, and packagedescription. In some cases, events that correspond to vectors that mostclosely match can be related to each other. In comparing contents of thedifferent arrays or vectors, the verification system 402 can rely on anormalization database, matching table, or algorithm. For example, toidentify related drugs, the verification system 402 can rely on anormalization database, such as the one described in greater detailabove with reference to FIGS. 6-8.

Furthermore, as described above, the verification system 402 can comparetime entries to identify related events. For example, the verificationsystem 402 can identify events that have one or more matching eventparameters and that are closest in time as related events.Alternatively, the verification system 402 can determine that events arenot related if the difference in time between events does not satisfy atiming threshold. For example, successive events that would otherwise berelated can be unrelated if they are more than a timing threshold (forexample, more than ten minutes, thirty minutes, or one hour) apart fromeach other. As another example, two or more events that are otherwiserelated events may be unrelated if the first event (for example, thefirst in time) of the otherwise related events and the last event (forexample, last in time) of otherwise related events are more than atiming threshold (for example, one or two hours) apart from each other.Further, in some cases, the verification system 402 can use sequencerules to identify related events. For example, the verification system402 can determine that a dispensing event is not related to anadministration event, a return event, or a waste event that precedes it.Additional rules and fuzzy logic can be used to identify related events.

From those related events, the verification system 402 can create orgenerate a plurality of event arrays. For example, each set of relatedevents can be aggregated to form a different event array such that asingle event array corresponds to a single set of related events. Insome cases, each event array can include only those events that arerelated to each other (as described above). Further, in some cases, noevents are included in more than one event array. As a non-limitingexample, a particular event array can include events that have amatching patient identifier, drug identifier, and location identifier.In some cases, events having matching patient identifiers, drugidentifiers, and location identifiers can be split into multiple eventarrays based at least in part on their timing identifiers.

In some cases, the verification system 402 can concurrently generate theevent arrays. Accordingly, discrepancies associated with one patient canbe determined in parallel with discrepancies in another patient. Forexample, in some cases, the verification system 402 can utilize one ormore processors to generate the event arrays in parallel, which canallow the system to efficiently identify discrepancies or potentialabuse.

At 5, the verification system 402 reconciles events and determinesdiscrepancies. For example, the verification system 402 can close outtransactions that have complete documentation (e.g., those combinationsof events that satisfy one or more rules applied thereto). To reconcileevents and determine discrepancies, the verification system 402 cancompare the data from the event arrays. If portions of the event arraycorrespond to each other, then the verification system 402 can reconcilethose portions. For example, for a particular event array, an entry mayindicate that a 20 mL vial of Propofol was dispensed and another,later-in-time entry can indicate that a 20 mL of Propofol wasadministered to a patient. In some cases, the verification system 402can reconcile those portions of data that resolve with each other (forexample, where the amount dispensed equals the amount administered plusamount discarded).

If one or more of the events of an event array do not reconcile withother events of the event array (for example, an amount dispensed doesnot equal an amount administered plus an amount wasted/returned), thenthe verification system 402 can determine that those events arediscrepancies. For example, for a particular event array, an entry mayindicate that a 20 mL vial of Propofol was dispensed and a related entryfrom the administration system 408 can indicate that a 15 mL of Propofolwas administered to a patient. Accordingly, the verification system 402can determine that 5 mL of Propofol is missing. Because 5 mL of Propofolis unaccounted for, the verification system 402 does not close out thoseentries. Instead, the verification system 402 can determine that thereis a discrepancy of 5 mL. However, if a subsequent entry is added to theevent array that indicates that 5 mL of Propofol was discarded, then theverification system 402 can reconcile those entries. Comparing theseentries and resolving any discrepancies can aid in preventing errors orabuse. Accordingly, by reconciling those entries in which the entireamount of the drug is accounted for. Further, by determining thoseinstances in which a discrepancy exists, the verification system 402 canidentify potential drug diverters, as described in more detail below. Itwill be understood that the verification system 402 can apply a varietyof rules to the event arrays to determine discrepancies as describedherein.

At 6A and 6B, the verification system 402 communicates discrepancies tothe dispensing system 404 and/or the administration system 408. In somecases, when a discrepancy is identified, the verification system 402 canalert a user. For example, the communication to the different systemscan include an indication that there is a discrepancy for a particularentry. Users of the different systems can use the indication to identifythe person that was in possession of the drug corresponding to thediscrepancy. In this way, the verification system 402 can help identifymissing drugs and prevent waste, fraud, and abuse. In addition, thenotification from the verification system 402 can cause the dispensingsystem 404 or the administration system 408 to correct a detected error,etc.

It will be understood that some of the data flow actions can occur in adifferent order or be omitted altogether. For example, the verificationsystem 402 can request and/or receive the administration data and/ordispensing data at different times, or the administration system 408 anddispensing system 404 can automatically report the relevant data withouta specific request from the verification system 402, or a user mayupload the data from the administration system 408 and dispensing system404 to the verification system 402. Further, multiple data requests toone system may correspond to a single request of another system. Forexample, the verification system 402 may make multiple requests for datafrom the administration system 408 to match up with data from thedispensing system 404 received after one request.

Further, fewer, more or different actions can be taken by theverification system 402. For example, the verification system 402 candisplay the discrepancies to a user. In addition, although illustratedas requesting/receiving information from two systems, it will beunderstood that the verification system 402 can interact with any numberof systems that include event data. For example, the verification system402 can receive relevant information about a pharmacy item, patient, oractivity from medical devices, waste systems, billing systems,scheduling systems, or from multiple administration systems 408 ordispensing systems 404, or inventory systems. In such embodiments, theverification system 402 can process the data from the various systems toidentify matching events from the different systems and track thelocation, use, and billing of the pharmacy items over time.

In some cases, any of steps 1-4 can occur automatically, without userinteraction. For example, in some cases, the event arrays are generatedautomatically and stored as data structures. Furthermore, any of steps 5or 6 can occur automatically or can be performed in response to or basedon a user's request. For example, a user can request information, and inreal-time, the system can parse all the relevant data in a time windowto identify and report discrepancies.

Obfuscating Personal Health Information

As mentioned herein, it can be difficult to communicate meaningfulinformation about events (for example, the dispensing of medication, theadministration of medication, etc.) without communicating personalhealth information (PHI). However, the communication of PHI is heavilyregulated.

FIG. 10 is a flow diagram illustrative of an embodiment of a routine1000 to communicate event data information between systems with the PHIobfuscated. One skilled in the relevant art will appreciate that theelements outlined for routine 1000 can be implemented using any one orany combination of the systems in the environment 400, such as theverification system 402, dispensing system 404, patient informationsystem 406, administration system 408, or billing system 410. Forsimplicity, the routine 1000 has been logically associated as beingimplemented by a system. However, the following illustrative embodimentshould not be construed as limiting. Furthermore, it will be understoodthat the various blocks described herein with reference to FIG. 10 canbe implemented in a variety of orders. For example, some blocks can beimplemented concurrently or the order can be changed, as desired.Furthermore, it will be understood that fewer, more, or different blockscan be used as part of the routine 1000. For example, in someembodiments, one or more of blocks 1002-1012 are not implemented.However, it will be understood that any of the blocks (and more ordifferent blocks) can be implemented as part of routine 1000.

At block 1002, the system receives a request for data that includes PHI.For example, the request for data can be similar to the request receivedby the dispensing system 404 at 1A of the flow diagram of FIG. 9 or therequest received by the administration system 408 at 1B of the flowdiagram of FIG. 9.

At block 1004, the system identifies PHI data for removal. In somecases, the system can identify PHI data that may not be used or requiredto respond to the request. For example, the system can identify that thepatient's name, social security number, gender, or the like is notrequired.

At block 1006, the system identifies PHI data for abstraction. Forexample, in some cases, some of the PHI data can be able to beabstracted to a range or otherwise adjusted or changed. For example, thesystem can identify numbers that can be converted into a range, such asa weight (for example, 175 lbs.) to a weight range (range, 160-180lbs.). As another example, the system can identify other information,such as time, that can be adjusted by a set amount. For example, if amedication was administered at 12:05 PM, it can be adjusted to 5:45 AM.In some cases, all systems that obfuscate PHI can adjust relatedinformation (for example, the time) by the same amount to ensureconsistency of the obfuscation.

At block 1008, the system removes the PHI data identified for removaland abstracts or changes the PHI data identified for abstraction. Insome cases, when multiple systems are communicating PHI data, each ofthe systems can be configured to remove and abstract the PHI data in thesame way for consistency purposes. Alternatively, in some cases, one ormore of the systems can abstract the PHI data in a different way, and acentralized or receiving system (for example, the verification system402) can be configured to associate the data from the different systemsdespite the different abstraction techniques.

In some cases, some of the data is neither removed nor abstracted. Forexample, the system can leave intact a user name (e.g., personperforming the action, such as the nurse or doctor), user identifier,user role (doctor, nurse, etc.), medication name, concentrationidentifiers, count (number of packages used), amount, transaction type(e.g., dispense, waste, return, administration, etc.) location, time,patient name, patient identifier, etc.

At block 1010, the system determines an obfuscated identifier for thedata to be transmitted. In some embodiments, the obfuscated identifiercan be randomly assigned. In certain embodiments, the obfuscatedidentifier can be determined based on the PHI data to be transmitted,such as a hash of the PHI data, or based on an identifier in the PHIdata, such as the name of the patient or patient ID.

In some cases, when different systems are determining the obfuscatedidentifier, they can determine the obfuscated identifier in the same waysuch that when the system that receives the data from other systems, itcan match the data. For example, a patient information system 406 anddispensing system 404 that have the same patient ID can transform orobfuscate the patient ID in the same way such that the obfuscatedidentifier received by a verification system 402 from both the patientinformation system 406 and dispensing system 404 are the same. In thisway, the verification system 402 can more easily match the data butremains unable to identify to whom the data corresponds.

In some cases, when different systems are determining the obfuscatedidentifier, each system determines the obfuscated identifier in adifferent way. As a result, it can be difficult for the system to matchthe data between the different systems. Thus, in some cases, the systemscan also communicate an indication of how the data was obfuscated. Forexample, a patient information system 406 and dispensing system 404 thathave the same patient ID can transform or obfuscate the patient ID in adifferent way, but can each communicate an indication of how the datawas obfuscated. In some cases, the communication of the indication ofhow the data was obfuscated can be encrypted or otherwise secure. Inthis way, the verification system 402 can determine how each systemobfuscated the data was obfuscated, but, in some cases, the data remainsunable to identify to whom the data corresponds.

At block 1012, the system communicates the data with the obfuscatedidentifier. In this way, the receiving system can perform variousanalytics or processing on the data without the PHI. In some cases, whenthe receiving system completes the processing it can return itsinformation along with the obfuscated identifier. In this way, thesystem can identify to which record the data from the other systemcorresponds and can update any records as desired.

It will be understood that the various blocks described herein withrespect to routine 1000 can be performed in a different sequence, can beadded, merged, or left out altogether, and that the routine 1000 canimplement one or more of the blocks concurrently and/or change theorder, as desired. Furthermore, it will be understood that fewer, more,or different blocks can be used as part of the routine 1000 tocommunicate event data information between systems with the PHIobfuscated. For example, in some embodiments, the system can performblock 1004 without or before receiving a data request. Furthermore, insome embodiments, the system can encrypt the data along with theobfuscated identifier to further reduce the likelihood of the PHI databeing discovered.

Event Arrays

FIG. 11 is a block diagram illustrating an embodiment of generation ofexample events arrays. With reference to FIG. 11, the events 1102 a,1102 b, 1102 c, 1102 d, and 1102 f can be normalized events and can eachinclude a plurality of event parameters including, but not limited to, afirst timing identifier 1104 a, a second timing identifier 1104 b, apatient identifier 1104 c, an action identifier 1104 d, a first drugidentifier 1104 e, and a second drug identifier 1104 f. From the events1102 a, 1102 b, 1102 c, 1102 d, and 1102 f, the verification system 402can generate the event arrays 1120 a, 1120 b, and 1120 c.

As described herein, the verification system 402 can receive events froma plurality of systems. In the illustrated embodiment, the plurality ofsystems includes a dispensing system 404, an administration system 408,and a waste system 1106. However, the verification system 402 canreceive events from fewer or more systems, as described herein.

Furthermore, as described herein, in some cases, the events receivedfrom the different systems 404, 408, and 1106 are not compatible witheach other. For example, the events from one system can be in adifferent format than another system. Accordingly, in some cases, theverification system 402 can normalize the events to format the eventssuch that they are in the same or a common format. For example, asillustrated, the verification system 402 can include a normalizationmodule 1110 that transforms the pre-processed events to processed ornormalized events. As described herein, the verification system 402 canuse a variety of techniques to normalize the events, including matchingdata from one system to normalized data or data from a normalizationdatabase, generating vectors of event parameters, and/or using cosinesimilarity to identify a normalized parameter to which the eventparameter corresponds.

In some cases, the verification system 402 can store the normalizedevents in a data store. For example, in some cases, as described herein,the verification system 402 will wait a predetermined period of timebefore generating the event arrays from the stored events. As such, theverification system 402 can aggregate and store the events until theverification system 402 is ready to process them to generate the eventarrays.

Using one or more of the event parameters 1104 a-1104 f, theverification system 402 can identify which, if any, of the events 1102a, 1102 b, 1102 c, 1102 d, and 1102 f are related. For example, asdescribed herein, the verification system 402 can identify that eventsare related based on a determination that the events include matchingpatient identifiers 1104 c, matching drug identifiers 1104 e, and/orsatisfy a timing threshold. With reference to the illustrated example ofFIG. 11, the verification system 402 can determine that events 1102 a,1102 d, and 1102 f are related. In addition, the verification system 402can determine that events 1102 b and 1102 c are each unrelated to anyother events.

Using the events and/or the event parameters, the verification system402 generates the event arrays 1120 a, 1120 b, and 1120 c based at leastin part on the identified related events. For example, as illustrated,event array 1120 a include related events 1102 a, 1102 d, and 1102 f. Insome cases, the event arrays 1120 a, 1120 b, and 1120 c can be generatedvia a two-step process. For example, as part of a first step in thegeneration of an event array, the verification system 402 can identify aset of normalized events that includes at least one matching eventparameter and/or can generate a first event array that includes theidentified set of normalized events. For example, in some cases, a firstevent array is generated that includes events with matching patientidentifiers 1104 c and drug identifiers 1104 e. As part of a second stepin the generation of an event array, the verification system 402 cangenerate a second event array from the first event array. For example,to generate the second event array, the verification system 402 canidentify, from the set of normalized events of the first array, thoseevents that occurred within a particular window of time from the one ormore of the events of the set of normalized events. For example, one ormore second event arrays can be generated from the first event array,where each second event array includes events that satisfy a timingthreshold and that include matching patient identifiers 1104 c and drugidentifiers 1104 e. With reference to the illustrated embodiment, theevent arrays 1120 a, 1120 b, and 1120 c can correspond to second eventarrays that satisfy a timing threshold and include matching patientidentifiers 1104 c and drug identifiers 1104 e. Although described as atwo-step process to generate an event array, it will be understood thatmore or fewer steps can be utilized. For example, the steps can becombined into a signal step. For example, the verification system 402can generate an event array by identifying a set of normalized eventsthat include at least one matching parameter and are within a particularwindow of time. Furthermore, in some cases, the process for generatingan event array can be different.

Although illustrated for five events, it will be understood that theverification system 402 can perform a similar event aggregation andevent array generation for more events from one or more systems. Forexample, the verification system 402 can concurrently generatenormalized events and perform a similar event aggregation and/or eventarray generation on thousands or millions of events. In some cases, bygenerating normalized events and then aggregating normalized events intoevent arrays, the system can significantly reduce the processing time togenerate event arrays. Furthermore, in some cases, the verificationsystem 402 can perform additional steps. For example, the verificationsystem 402 can remove duplicates from the events 1102 a-1102 bycomparing event parameters of the events to determine if any matchesexist. If a match exists, the verification system 402 can remove one orthe events corresponding to the match. In addition, although illustratedas various event parameters 1104 a-114 f are shown, it will beunderstood that the events can include fewer or more event parameters.

Furthermore, with respect to generation of the event arrays 1120 a-1120c, it will be understood that these event arrays can be generatedconcurrently by the verification system 402. In some cases, as describedherein, the verification system 402 can wait a predetermined period oftime before generating the event arrays 1120 a-1120 c using the events(or normalized events) in the data store. For example, as illustrated,while event array 1120 a may satisfy one or more rules (e.g., dispenseamount equals administration and waste amounts), event arrays 1120 b and1120 c may not. That is, in the illustrated example, event arrays 1120 band 1120 c each include a dispense event, but there is no correspondingadministration, waste, or return event. In some cases, to increase thelikelihood that the verification system 402 has received all of therelated events for an event array, the verification system 402 can waita predetermined period of time after a time stamp associated with anevent to generate an event array.

For example, event 1102 f has a timing identifier 1104 b of 11:42:48.Accordingly, in some cases, the system can wait a predetermined periodof time (for example, 1 hour) before generating the event array 1120 a.In some cases, the verification system 402 can query the systems at oraround 12:42:48 to increase the likelihood that is has received therelevant events.

In some cases, the predetermined period of time over which the systemwaits to generate the summary can correspond to a timing identifier ofone or more events that are determined to be related events. Forexample, as described above, events 1102 a, 1102 d, and 1102 f arerelated events. In some cases, the system can wait a predeterminedperiod of time after the time stamp of the first event of the relatedevents (in this case, event 1102 a) before generating the event arrayfor that event. In some cases, the system can wait a predeterminedperiod of time after the time stamp of the last event of the relatedevents (in this case, event 1102 f) before generating the event arraythat includes for that event.

It will be understood that although the verification system 402 canconcurrently generate the event arrays, as described above, in somecases, the generation of one or more event arrays can be paused or puton hold until the system waits the predetermined period of timecorresponding to that event or group of related events. Thus, althoughthe system waits to generate an event array for some events, it canproceed to generate event arrays for other events that satisfy thetiming threshold has been satisfied.

Event Tracking and Diversion Detection

Pharmacies, hospitals, and other medical facilities are continuouslyconfronting the increasing challenge of theft of controlled substances,also referred to as drug diversion. Drug diversion can occur anywhere inthe distribution, administration, and documentation chain. While medicalfacilities are under pressure to account for and control losses, thosewho divert or steal drugs (generally referred to as diverters) can bedifficult to detect. Accordingly, medical facilities need better toolsto identify and prevent drug diversion, while remaining compliant withprivacy related regulations.

As described herein, a system, such as verification system 402, canassociate data between various systems (including but not limited to thedispensing system 404, the patient information system 406, theadministration system 408, or the billing system 410), reconcile events,and identify discrepancies between events. By reconciling or closing outtransactions that have complete documentation of the drug, andidentifying discrepancies, the system can focus on specific events (forexample, those associated with the discrepancies) to help to minimizerisk of future diversions.

FIG. 12 is a flow diagram illustrative of an embodiment of a routine1200 to determine event discrepancies between disparate systems. Oneskilled in the relevant art will appreciate that the elements outlinedfor routine 1200 can be implemented using any one or any combination ofthe systems in the environment 400, such as the verification system 402,dispensing system 404, patient information system 406, administrationsystem 408, or billing system 410. For simplicity, the routine 1000 hasbeen logically associated as being implemented by the verificationsystem 402. However, the following illustrative embodiment should not beconstrued as limiting. Furthermore, it will be understood that thevarious blocks described herein with reference to FIG. 12 can beimplemented in a variety of orders. For example, some blocks can beimplemented concurrently or the order can be changed, as desired.Furthermore, it will be understood that fewer, more, or different blockscan be used as part of the routine 1200. For example, in someembodiments, one or more of blocks 1202-1216 are not implemented.However, it will be understood that any of the blocks (and more ordifferent blocks) can be implemented as part of routine 1200.

At block 1202, the verification system 402 queries a plurality ofsystems for data related to one or more events. The plurality of systemscan include at least a first system and a second system, and one or bothof the first system or the second system can include one or more of thedispensing system 404, the patient information system 406, theadministration system 408, or billing system 410, as described herein.

To query the systems, the verification system 402 can transmit one ormore requests for the data via a network, such as network 412 of FIG. 4.For example, the verification system 402 can send a request thatrequests a system to update the verification system 402 with theparticular system's data that is related to one or more events. Inaddition or alternatively, the request can include a request for datarelated to one or more events that occurred within a particular timewindow, such as within the past hour, day, or week, or with in aparticular time or date range.

In some cases, the verification system 402 can query the systems basedat least in part on a user command. For example, a user can submit arequest for the data (non-limiting examples: by opening a web browser orapplication, by interacting with a graphical user interface), andresponsive to the request by user, the verification system 402 can querythe systems. In addition or alternatively, the verification system 402can query the plurality of systems automatically, such as at one or moretime intervals. For example, the verification system 402 can query thesystems one or more times a second or every few seconds. In addition oralternatively, the verification system 402 can query the systems hourly,daily, weekly, monthly, or the like. In some cases, the interval atwhich the verification system 402 queries the systems can beconfigurable by a user. Moreover, in some cases, the verification system402 can query one or more of the systems at different times orsimultaneously. Furthermore, as described in more detail below, in somecases, one or more systems can provide data to the verification system402 without receiving a request from the verification system 402. Forexample, the one or more systems can be configured to send the data tothe verification system 402 at one or more time intervals, and theverification system 402 can passively receive the data.

At block 1204, the verification system 402 receives a first set ofevents from the first system. The first set of events can include datarelated to one or more events including, but not limited to, adispensing event, an administration event, a waste event, or a returnevent. For example, each event of the first set of events can include aplurality of event parameters. The event parameters include acombination of one or more of an action identifier (non-limitingexamples: identifying the event as a dispensing event, an administrationevent, a waste event, or a return event), drug data (drug name, drugquantity, drug concentration, package size, or other drug identifyinginformation), a patient identifier, a time identifier, a care areaidentifier, a provider identifier (for example, name of the caregiver,nurse, or doctor performing the event), a witness identifier(non-limiting example: a name of a witness of the event), or the like.

At block 1206, the verification system 402 receives a second set ofevents from a second system. As described above with respect to thefirst set of events, the second set of events can include data relatedto one or more events including, but not limited to, a dispensing event,an administration event, a waste event, or a return event. Furthermore,each event of the second set of events can include a plurality of eventparameters, as described above with respect to block 1204.

In some cases, the second set of events received at block 1206 aredifferent from and are not compatible with the first set of eventsreceived at block 1204. For example, although the events of the firstand second sets of events can each include event parameters, the firstand second sets of events, or the event parameters, can be differentfrom and/or incompatible with each other. For example, the first andsecond sets of events can be stored in or provided by the systems asdifferent file types or different versions of a similar file type. Insome cases, the first and second sets of events can be stored in thesame or a similar file type, yet the event parameters associated witheach of the sets of events are different and/or are stored or sorted ina different order.

Although the routine 1200 includes the verification system 402 receivingsets of events from the first and second systems, the verificationsystem 402 can additionally or alternatively receive sets of events fromother systems.

At block 1208, the verification system 402 normalizes the first set ofevents and the second set of events. For example, to normalize the firstset of events and the second set of events (and any other sets ofevents), the verification system 402 can convert the sets of events intoprocessed events or normalized events as described herein.

In some cases, as part of the normalization of the first set of eventsand the second set of events, the verification system 402 can aggregatethe events such that some or all of the normalized events of the firstset of events and the second set of events are collected or combinedinto an array. Moreover, as described herein, in some cases, theverification system 402 can remove duplicate events from the aggregatedevents.

At block 1210, the verification system 402 concurrently generates aplurality of event arrays. For example, in some cases, the verificationsystem 402 can utilize a plurality of processors to generate the eventarrays in parallel. As described herein, the verification system 402 cancompare the contents of the different events to identify and/or grouprelated events. As a non-limiting example, in certain cases, theverification system 402 can determine that events are related if theevents are associated with the same patient identifier, drug identifier,location identifier, and/or satisfy a timing threshold. Each set ofrelated events can be aggregated to form an event array (first or secondevent array as described herein), such that a single event arraycorresponds to a single set of related events. In some cases, each eventarray can include (only) those events that are related to each other (asdescribed above). Further, in some cases, no events are included in morethan one event array. However, in certain cases, events can be includedin multiple arrays. For example, an event can be included in a firstevent array based on its patient identifier and drug identifier and canbe included in a second event array based on its patient identifier,drug identifier, and timing identifier.

At block 1212, the verification system 402 can apply at least one ruleto the event array to generate one or more event array parameters. Forexample, the rule can include one or more instructions that specifies orindicates how to generate an event array parameter from the event array.In general, a rule can include an instruction for extracting a value(e.g., an event array parameter) from or determining a value associatedwith the at least one event array. In some cases, the applied rule caninclude one or one of a sequence rule, dispensing rule, omission rule,timing rule, drug quantity rule, location rule, witness rule, orbehavior rule, described in more detail below, or other rule as the casemay be.

At block 1214, the verification system 402 compares the event arrayparameter to one or more corresponding event array parameter thresholds.As described in more detail below, an event array parameter thresholdcan come in many forms, and can be based at least in part on theapplied.

In some cases, the event array parameter threshold can correspond to abinary output, such as yes or no, true or false, 1 or 0, high or low,etc. As a non-limiting example, the event array parameter threshold canbe “yes.” Accordingly, in some embodiments, if the event array parametercorresponds to “yes,” then the system can determine that the event arrayparameter satisfies the event array parameter threshold, and if theevent array parameter is “no,” then the system can determine that theevent array parameter does not satisfy the event array parameterthreshold, or vice versa.

In some cases, the event array parameter threshold can correspond to astring or list. In cases such as these, if the event array parameterincludes a string or list that matches (or substantially matches) thestring or list of the event array parameter threshold, then the systemcan determine that the event array parameter satisfies the event arrayparameter threshold. In contrast, if the event array parameter does notinclude a string or list that matches the string or list of the eventarray parameter threshold, then the system can determine that the eventarray parameter does not satisfy the event array parameter threshold.

As another example, the event array parameter threshold can correspondto a value. As a non-limiting example, the event array parameterthreshold can be 30. Accordingly, in some cases, if the event arrayparameter is greater than or equal to 30, then the system can determinethat the event array parameter satisfies the event array parameterthreshold. Alternatively, in some cases, if the event array parameter isless than or equal to 30, then the system can determine that the eventarray parameter satisfies the event array parameter threshold.

The verification system 402 can determine event array parameterthresholds using a variety of techniques. For example, in some cases,the verification system 402 can determine the event array parameterthresholds based at least in part on a set of event arrays. The set ofevent arrays can be selected based on a window of time, similaritybetween event parameters (e.g., position or user ID, etc.) Accordingly,what is considered normal (for example, satisfies a threshold) from oneset of event arrays can be abnormal (or fails to satisfy a threshold) inanother.

In some cases, the verification system 402 can determine an average(non-limiting examples: average time between events of the event arrays,average or most common sequence of events in the event arrays, commonevents in event arrays, etc.) for the set of event arrays and determinethe event array parameter threshold based on a deviation from theaverage (non-limiting example: standard deviation, difference, etc.

At block 1216, the verification system 402 can cause an indication tothe user that at least one event array parameter satisfies itsrespective event array parameter threshold and/or an indication that atleast one array parameter does not satisfy its respective event arrayparameter threshold. For example, as described herein at least withrespect to FIG. 13, in some cases, the system can display a graphicaluser interface that displays the indication.

It will be understood that the various blocks described herein withrespect to routine 1200 can be performed in a different sequence, can beadded, merged, or left out altogether, and that the routine 1200 canimplement one or more of the blocks concurrently and/or change theorder, as desired. Furthermore, it will be understood that fewer, more,or different blocks can be used as part of the routine 1200 to determineevent discrepancies between disparate systems.

For example, the verification system 402 can receive the first set ofevents and/or the second set of events without querying systems for thedata. For example, in some cases, one or more systems can be configuredto transmit the events to the verification system 402 at predeterminedintervals. Furthermore, in some embodiments, the verification system 402can sort the plurality of normalized events based on one or more of theevents parameters associated with an event. For example, theverification system 402 can sort the plurality of normalized eventsbased on the timing identifier. Furthermore, as described in more detailbelow, in some embodiments, the verification system 402 can generateand/or display a score based on the comparison of the event arrayparameter and the event array parameter thresholds.

Rules and Event Array Parameters

As described herein, the verification system 402 can apply one or morerules to the event arrays. Based on the rules applied to the eventarrays, the verification system 402 can generate one or more event arrayparameters. The event array parameters can be compared with event arrayparameter thresholds. Based on the comparison, the verification system402 can indicate a potential misuse or abnormality.

In some cases, the verification system 402 can generate the event arrayparameter for each event array. For example, the verification system 402can generate one event array parameter for each event array by applyinga sequence rule and another event array parameter for each event arrayby applying an omission rule.

In certain cases, the verification system 402 generates an event arrayparameter for a combination of event arrays, such as a group of eventarrays associated with the same user. For example, the verificationsystem 402 can generate an event array parameter for a group of eventarrays associated with a particular user by applying a timing rule tothe event arrays associated with the user. As a specific example, theverification system 402 can generate an average time between events ofevent arrays associated with the user.

Event Array Parameter Thresholds

For each rule applied to one or more event arrays, the verificationsystem 402 can determine or use an event array parameter threshold. Theevent array parameter thresholds can be static or dynamically determinedby the verification system 402. For example, in some cases, thedifference between an amount dispensed and the combination of the amountadministered, wasted, and returned is to be zero. Thus, in certaincircumstances, the verification system 402 can use zero as the eventarray parameter threshold when analyzing event arrays using a dispensingrule.

In cases where the event array parameter threshold is dynamic, theverification system 402 can determine the event array parameterthreshold using a variety of techniques. For example, in some cases, theverification system 402 can determine an event array parameter thresholdbased at least in part on a set of event arrays. The set of event arrayscan be selected based on a window of time, similarity between eventparameters (e.g., position or user ID, etc.), etc. Thus, the event arrayparameter thresholds can vary from one set of event arrays to another.

In some cases, the verification system 402 can determine an average(non-limiting examples: average time between events of the event arrays,average or most common sequence of events in the event arrays, commonevents in event arrays, average number of event arrays for users with adiscrepancy or that do not satisfy an event array parameter thresholdassociated with an omission or dispensing rule, etc.) for the set ofevent arrays and determine the event array parameter threshold based ona deviation from the average (non-limiting example: standard deviation,difference, etc.

Rule Thresholds

As described herein, the system can apply one or more rules to anindividual event array to determine whether the event array satisfies anevent array parameter threshold. Similarly, in some embodiments, theverification system 402 can use multiple event arrays to determinewhether a user satisfies a rule threshold. In some cases, thesatisfaction of an event array threshold by a single event array canindicate misuse. In certain cases, it is multiple event arrayssatisfying multiple event array thresholds that indicate misuse.Accordingly, the verification system 402 can aggregate event arraysassociated with a user to determine the likelihood of misuse. In certaincases, the verification system 402 can use a rule threshold to determinethe likelihood of misuse.

In some cases, the rule threshold can be based on a set of data. Forexample, the verification system 402 can analyze a set of data todetermine an average number of event arrays that satisfy different eventarray parameter thresholds. Based on the determined average, theverification system 402 can determine the rule threshold. For example,the rule threshold can be determined by identifying a deviation from theaverage, such as a difference from the average or a standard deviationfrom the average.

The system can determine a threshold for each of the rules applied tothe event arrays. For example, the system can determine a sequence rulethreshold, a dispensing rule threshold, omission rule threshold, timingrule threshold, drug quantity rule threshold, location rule threshold,witness rule threshold, or behavior rule threshold. Based on the rulethresholds, the verification system 402 can identify one or moreabnormalities associated with a particular user.

Sequence Rules

In some cases, one or more sequence rules can be applied to some or allevent arrays to generate one or more event array sequence parameters. Asequence rule can include instructions to identify various eventsequences corresponding to the event array(s). The application of thesequence rule (for example, the result of performing the instructions)can result in an event array sequence parameter.

In general, each event array, regardless of the number of events withinthe event array, can be associated with an overall sequence. As anon-limiting example, an event array can include three events, and, fromearliest in time to latest in time, the events can include a dispensingevent, administration event, and a waste event. Here, the overallsequence of the event array can include a list, a string, or otherindication of the order of the events: that is,dispense-administration-waste. Accordingly, in cases in which thesequence rule includes instructions to identify the overall sequence ofevents, the event array sequence parameter can include a list, a string,or other indication of the order of the events.

In many instances, in addition to the overall sequence, multiple othersequence identifiers can be extracted from a particular event array. Forexample, a sequence rule can include instructions to identify a firstevent in the sequence of events. Accordingly, in this case, theresulting event array sequence parameter can include an indication of adispense event. Similarly, a sequence rule can include instructions toidentify a last event in the event array. Accordingly, in this case, theresulting event array parameter can include an indication of a wasteevent.

In addition or alternatively, a sequence rule can include instructionsto identify whether a particular type of event follows or precedesanother particular type of event. In instances such as these, the eventarray sequence parameter can be a selection of a binary response(non-limiting examples: yes/no, true/false, 1/0, high/low, etc.). Forexample, a sequence rule can include instructions to identify whether anadministration event immediately follows (or is the next event in thesequence) a dispense event. In this case, the resulting event arraysequence parameter can include an indication of “yes,” since theadministration event is the next event following the dispense event. Itwill be understood that similar sequence rules can be applied to theevent array(s) to generate various event array parameters. Further itwill be understood that the event array sequence parameter can includevarious data, which can depend on the applied rule.

In some cases, an event array sequence parameter can be compared to oneor more event array sequence thresholds to determine whether the eventarray sequence parameter satisfies the respective event array sequencethreshold. In some cases, if the event array sequence parametersatisfies the event array sequence threshold, then the particularsequence of events taken by a provider can be considered normal, usual,acceptable, or the like. In contrast, in some cases, if the event arraysequence parameter does not satisfy the event array sequence threshold,then the particular sequence of events taken by a provider can beconsidered abnormal, unusual, unacceptable, or the like.

As described herein, the event array sequence thresholds can come inmany forms and can be based at least in part on the applied sequencerule. For example, if the event array parameter includes a binaryoutput, as described above, then the event array parameter threshold canalso include a binary output. For example, if the sequence rule includesinstructions to identify whether an administration event immediatelyfollows a dispense event, then the event array sequence threshold can be“yes.” Accordingly, in this example, if the event array sequenceparameter is “yes” then the event array parameter satisfies the eventarray parameter threshold. However, in this example, if the event arrayparameter is “no” then the event array parameter does not satisfy theevent array parameter threshold.

As another example, the event array parameter threshold can include astring, a list, or other indication of the sequence of events. In casessuch as these, the event array parameter can satisfy the event arrayparameter threshold if the event array parameter matches (e.g., includesthe same a string, a list, or other indication of the sequence ofevents) the event array parameter threshold. In contrast, in some cases,the event array parameter does not satisfy the event array parameterthreshold if the event array parameter does not match the event arrayparameter threshold.

In some cases the event array sequence threshold can correspond to aprobability of a particular sequence occurring. For example, theverification system 402 can build, populate, or obtain a variable-lengthMarkov model, which describes the probabilities of different sequencesof actions occurring (for example, for a given medication), based on aset of data. For example, the system 402 can determine the probabilityof a first event in a sequence being a dispense event in a set of data,and the event sequence threshold can correspond to that probability. Asanother example, the system can determine how likely it is that a wasteevent is preceded by a dispensing event and/or an administration event.In some cases, the event array sequence thresholds can be generatedusing a set of event arrays, such as those event arrays associated witha particular provider, all providers, or a subset of providers, such asproviders with a similar access level.

As a non-limiting example, probability data may indicate that adispensing event is the first event in a sequence 88% of the time, andan administration event is the first event in a sequence only 11.5% ofthe time. Furthermore, the probability data may indicate that, followinga dispense event as a first event, in 99.2% of cases, the next event isan administration event. If the sequence rule includes an instruction toidentify whether a dispense event is the first event, and, if so,whether an administration event is the second event, then the eventarray parameter can include a binary indication, such as yes or no.Based on probability, if the dispense event is the first event, then anadministration event can be expected to be the second event.Accordingly, in this case, the event array parameter can satisfy theevent array parameter threshold if the event array parameter is yes.Otherwise, in some cases, the event array parameter does not satisfy theevent array parameter threshold. Accordingly, in some cases, thesequence rules and/or the resulting event array parameters can beindicative of whether an individual deviated from the norm or wasunusual as compared to others.

As described herein, in addition, to determining whether an individualevent array satisfies the event array sequence threshold, theverification system 402 can track the number of event arrays associatedwith a user that do or do not satisfy the event array sequencethreshold. This number can be compared with a sequence rule threshold todetermine whether the provider satisfies a sequence rule threshold.

The verification system 402 can determine the sequence rule thresholdbased on individual event array sequence parameters of the event arraysof a set of data. For example, the verification system 402 can determinean average number of event arrays that satisfy the event array sequencethreshold for individual users. A deviation or different from theaverage can then be used as the sequence rule threshold. If the averagenumber of event arrays for a particular user satisfies the event arraysequence threshold, the verification system 402 can identify the user asan abnormal user or otherwise indicate that the user does not satisfythe sequence rule threshold.

Dispensing Rules

In some cases, one or more dispensing rules can be applied to some orall event arrays to generate one or more event array dispensingparameters. In some embodiments, a dispensing rule can includeinstructions to identify whether one type of event follows another typeof event, such as whether an administration event follows eachdispensing event in the event array. The application of the dispensingrule (for example, the result of performing the instructions) can resultin an event array dispensing parameter.

As a non-limiting example, an event array can include two events. Fromearliest in time to latest in time, the event array can include adispensing event and a waste event (but no administration event).Accordingly, in this case, the event array dispensing parameter caninclude a binary output of “no.” However, if the event array included athird event which corresponded to administration, then the event arraydispensing parameter can include a binary output of “yes.”

In some cases, the event array dispensing parameter can be compared toone or more event array dispensing thresholds to determine whether theevent array parameter satisfies the respective event array dispensingthreshold. In some cases, if the event array dispensing satisfies theevent array dispensing threshold, then the event can be considerednormal, usual, acceptable, or the like. In contrast, in some cases, ifthe event array dispensing parameter does not satisfy the event arraydispensing threshold, then the event can be considered abnormal,unusual, unacceptable, or the like.

As described herein, the event array dispensing thresholds can come inmany forms and can be based at least in part on the applied rule. Forexample, if the event array parameter includes a binary output, asdescribed above, then the event array dispensing threshold can alsoinclude a binary output. For example, if the dispensing rule includesinstructions to identify whether an administration event follows eachdispensing event in the event array, then the event array parameterthreshold can be “yes.” Accordingly, if the event array parameter is“yes” then the event array parameter satisfies the event array parameterthreshold. However, if the event array parameter is “no” then the eventarray parameter does not satisfy the event array parameter threshold.

As described above, in some cases, it can be unusual to dispense thedrug without administering that drug to the patient. Accordingly, if theevent array parameter does not satisfy the event array parameterthreshold, then the system can identify the event array, or the provideras being associated with abnormal behavior, which may be the result ofdrug misuse.

As described herein, in addition, to determining whether an individualevent array satisfies the event array dispensing threshold, theverification system 402 can track the number of event arrays associatedwith a user that do or do not satisfy the event array dispensingthreshold. This number can be compared with a dispensing rule thresholdto determine whether the provider satisfies a dispensing rule threshold.

The verification system 402 can determine dispensing rule thresholdbased on individual event array dispensing parameters of the eventarrays of a set of data. For example, the verification system 402 candetermine an average number of event arrays that satisfy the event arraydispensing threshold for individual users. A deviation or different fromthe average can then be used as the dispensing rule threshold. If theaverage number of event arrays for a particular user satisfies the eventarray dispensing threshold, the verification system 402 can identify theuser as an abnormal user or otherwise indicate that the user does notsatisfy the dispensing rule threshold.

Omission Rules

In some cases, one or more omission rules can be applied to some or allevent arrays to generate one or more event array parameters. An omissionrule can include instructions to identify whether the following equationis satisfied:

Dispensed Amount=Administered Amount+Wasted Amount+Returned Amount  (Equation 1)

where, when considering all events of a particular event array,“dispensed” is the total amount of a particular medication that wasdispensed (for example, from the dispensing system 404) for a particularpatient, “administered” is the total amount of the particular medicationthat was identified as administered to the particular patient (forexample, by the administration system 408), “wasted” is the total amountof the particular medication that was identified as discarded (forexample, by the dispensing system 404), and “returned” is the totalamount of the particular medication that was identified as returned (forexample, by the dispensing system 404).

In some embodiments, the application of the omission rule (for example,the result of performing the instructions) can result in the event arrayparameter. As a non-limiting example, if 20 mL of Diprivan is dispensedfor Patient A, but only 15 mL of Diprivan is administered to Patient A,and no Diprivan is wasted or returned, then the system can identify thatEquation 1 is not satisfied because 5 mL of Diprivan is not accountedfor. Accordingly, in this case, the event array parameter can include abinary output of “no.” In some cases, as described herein, event arraysin which at least some of the medication is not accounted for (that is,the group of entries does not satisfy Equation 1), can be classified asdiscrepancies. However, if the event array included a fourth event whichcorresponded to a return event of 5 mL of Diprivan, then the event arrayparameter can include a binary output of “yes.”

In some embodiments, the event array parameter and event array parameterthreshold can be values. For example, the event array parameterthreshold can be zero indicating that the difference between the amountdispensed and the combination of the amount administered, returned andwasted is to be zero. In certain cases, the event array parameterthreshold can be non-zero, such as one or two indicating that thedifference between the amount dispensed and the combination of theamount administered, returned and wasted should be less than one or two.

In some cases, the event array parameter can be compared to one or moreevent array parameter thresholds to determine whether the event arrayparameter satisfies the respective event array parameter threshold. Insome cases, if the event array parameter satisfies the event arraythreshold, then the events taken by a provider can be considered normal,usual, acceptable, or the like. In contrast, in some cases, if the eventarray parameter does not satisfy the event array threshold, then theevents taken by a provider can be considered abnormal, unusual,unacceptable, or the like.

As described herein, the event array parameter thresholds can come inmany forms, and can be based at least in part on the applied rule. Forexample, if the event array parameter includes a binary output, asdescribed above, then the event array parameter threshold can alsoinclude the binary output. For example, if the omission rule includesinstructions to identify whether Equation 1 is satisfied, then the eventarray parameter threshold can be “yes.” Accordingly, if the event arrayparameter is “yes” then the event array parameter satisfies the eventarray parameter threshold. However, if the event array parameter is “no”then the event array parameter does not satisfy the event arrayparameter threshold.

As described herein, in addition to determining whether an individualevent array satisfies the event array omission threshold, theverification system 402 can track the number of event arrays associatedwith a user that do or do not satisfy the event array omissionthreshold. This number can be compared with an omission rule thresholdto determine whether the provider satisfies an omission rule threshold.

The verification system 402 can determine the omission rule thresholdbased on individual event array omission parameters of the event arraysof a set of data. For example, the verification system 402 can determinean average number of event arrays that satisfy the event array omissionthreshold for individual users. A deviation or difference from theaverage can then be used as the omission rule threshold. If the averagenumber of event arrays for a particular user satisfies the event arrayomission threshold, the verification system 402 can identify the user asan abnormal user or otherwise indicate that the user does not satisfythe omission rule threshold. For example, for a set of data, if theaverage number of event arrays for each user that satisfy the eventarray omission threshold is three with an event array omission thresholdof ±2 from the average, and one user has six event arrays that satisfythe event array omission threshold, then the verification system 402 candetermine that the user does not satisfy the omission rule threshold,and indicate the same to a user of the verification system 402.

Timing Rules

In some cases, the duration over which a user has control of or isholding onto a drug can be indicative of drug diversion or drug misuse.Accordingly, in some cases, one or more timing rules can be applied toan event array to generate an event array timing parameter. A timingrule can include instructions to identify a duration between to two ormore events of an event array, and the application of the timing rule(for example, the result of performing the instructions) can result inthe event array timing parameter.

As a non-limiting example, an event array can include three events: afirst event that occurred at 12:00 PM, a second event that occurred at12:10 PM, and a third event that occurred at 1:05 PM. Here, thedifference in time between the first and second events, is 10 minutes,the difference in time between the first and last events, is 1 hour 5minutes, and the difference in time between the second and third events,is 55 minutes. Accordingly, in some cases, the event array timingparameter can include a numerical value, such as an indication of thedifference in time between two events. In addition or alternatively, theevent array timing parameter can include a binary output, such as yes orno. For example, a timing rule can include instructions to identifywhether a duration between to two or more events of an event array isgreater than or less than a predetermined value.

In some cases, the event array timing parameter can be compared to oneor more event array timing thresholds to determine whether the eventarray timing parameter satisfies the respective event array timingthreshold. In some cases, if the event array timing parameter satisfiesthe event array threshold, then the events taken by a provider can beconsidered normal, usual, acceptable, or the like. In contrast, in somecases, if the event array timing parameter does not satisfy the eventarray threshold, then the events taken by a provider can be consideredabnormal, unusual, unacceptable, or the like.

As described herein, the event array timing thresholds can come in manyforms and can be based at least in part on the applied rule. Forexample, if the event array timing parameter includes a binary output,as described above, then the event array timing threshold can alsoinclude the binary output. For example, if the timing rule includesinstructions to identify whether a duration between to two or moreevents of an event array is greater or less than a predetermined value,the event array timing threshold can be “yes” or “no.” In some cases, ifthe event array timing parameter is “yes,” then the event array timingparameter satisfies the event array timing threshold. Similarly, in somecases, if the event array timing parameter is “no” then the event arraytiming parameter does not satisfy the event array timing threshold.

As another example, the event array timing threshold can include anumber, such as a number of minutes or hours. In cases such as these, insome instances, the event array timing parameter can satisfy the eventarray timing threshold if the event array timing parameter is greaterthan the event array timing threshold. Alternatively, in some instances,the event array timing parameter can satisfy the event array timingthreshold if the event array timing parameter is less than the eventarray timing threshold. Furthermore, as described herein, the eventarray timing threshold can be determined based on a set of event arrays.For example, the verification system 402 can determine an average timebetween two events of a set of event arrays, identify the average plusor minus a certain amount (predetermined amount, standard deviation,etc.) as the event array timing threshold, and compare the time betweentwo events of a particular event array with the determined event arraytiming threshold.

As described herein, in addition to determining whether an individualevent array satisfies the event array timing threshold, the verificationsystem 402 can track the number of event arrays associated with a userthat do or do not satisfy the event array timing threshold. This numbercan be compared with a timing rule threshold to determine whether theprovider satisfies a timing rule threshold.

The verification system 402 can determine the timing rule thresholdbased on individual event array timing parameters of the event arrays ofa set of data. For example, the verification system 402 can determine anaverage number of event arrays that satisfy the event array timingthreshold for individual users. A deviation or different from theaverage can then be used as the timing rule threshold. If the averagenumber of event arrays for a particular user satisfies the event arraytiming threshold, the verification system 402 can identify the user asan abnormal user or otherwise indicate that the user does not satisfythe timing rule threshold. For example, for a set of data, if theaverage time between two events is two minutes with a standard deviationof thirty seconds (timing rule threshold of ±1 minute from average) anda user's average time between the two events is twenty seconds or fiveminutes, then the verification system 402 can determine that the userdoes not satisfy the timing rule threshold, and indicate the same to auser of the verification system 402.

Drug Quantity Rules

In some cases, the amount of a drug dispensed by a user be indicative ofdrug misuse. Accordingly, in some cases, one or more drug quantity rulescan be applied to an event array to generate an event array quantityparameter. A drug quantity rule can include instructions to identifyvarious drug quantities corresponding to the event array, and the resultof the application of the drug quantity rule (for example, the result ofperforming the instructions) is the event array parameter.

As a non-limiting example, an event array can include three events: adispensing event of 30 mg is associated with Provider A, a dispensingevent of 20 mg is associated with Provider B, an administration event of20 mg is associated with Provider A, a return event of 10 mg isassociated with Provider A, and a return event of 20 mg is associatedwith Provider B. In some cases, a drug quantity rule includesinstructions to identify the total amount of drug that is dispensed by aparticular provider. In this example, the Provider A dispensed 30 mg,while Provider B dispensed 20 mg. In some cases, a drug quantity ruleincludes instructions to identify the total amount of drug that isadministered by a particular provider. In this example, the Provideradministered 20 mg, while Provider B administered 0 mg. In some cases, adrug quantity rule includes instructions to identify the total amount ofdrug that is wasted by a particular provider. In this example, theProvider wasted 10 mg, while Provider B wasted 20 mg. In some cases, adrug quantity rule includes instructions to identify the total amount ofdrug that is returned by a particular provider. In this example, theProvider returned 0 mg, while Provider B returned 0 mg. Accordingly, insome cases, the event array quantity parameter can include a number orindication of an amount of a drug.

In addition or alternatively, a drug quantity rule can includeinstructions to identify whether a particular administrator has used(for example, dispensed, administered, discarded or returned) athreshold amount of a drug. In instances such as these, the event arrayquantity parameter can be a selection of a binary response (non-limitingexamples: yes/no, true/false, I/O, high/low, etc.). For example, a drugquantity rule can include instructions to identify whether a providerhas dispensed at least a threshold amount (for example, 15 mg) of aparticular drug. In this case, the resulting event array quantityparameter or an analysis of both Provider A and Provider B can includean indication of “yes,” since each dispensed greater than 15 mg.Similarly, a drug quantity rule can include instructions to identifywhether a particular administrator has not used (for example, dispensed,administered, discarded or returned) a threshold amount of drug. Ininstances such as these, the event array quantity parameter can be aselection of a binary response (non-limiting examples: yes/no,true/false, I/O, high/low, etc.). For example, a drug quantity rule caninclude instructions to identify whether a provider has not dispensed atleast a threshold amount (for example, 25 mg) of a particular drug. Inthis case, the resulting event array quantity parameter for an analysisProvider A can include an indication of “yes” since Provider A dispensedgreater than 25 mg. However, the resulting event array quantityparameter for an analysis Provider B can include an indication of “no”since Provider B did not dispense greater than 25 mg. It will beunderstood that similar drug quantity rules can be applied to the eventarray(s) to generate various event array parameters. Further it will beunderstood that the event array quantity parameter can include variousdata, which can depend on the applied rule.

In some cases, the event array quantity parameter can be compared to oneor more event array quantity thresholds to determine whether the eventarray quantity parameter satisfies the respective event array quantitythreshold. In some cases, if the event array quantity parametersatisfies the event array threshold, then the events taken by a providercan be considered normal, usual, acceptable, or the like. In contrast,in some cases, if the event array quantity parameter does not satisfythe event array threshold, then the events taken by a provider can beconsidered abnormal, unusual, unacceptable, or the like.

The event array quantity thresholds can come in many forms and can bebased at least in part on the applied rule. For example, if the eventarray quantity parameter can include a binary output, as describedabove, then the event array quantity threshold can also include thebinary output. For example, if the drug quantity rule includesinstructions to identify whether a provider has dispensed at least athreshold amount, then the event array quantity threshold can be “yes”or “no.” If the event array quantity parameter is “yes,” then the eventarray quantity parameter satisfies the event array quantity threshold.However, if the event array quantity parameter is “no” then the eventarray quantity parameter does not satisfy the event array quantitythreshold.

As another example, the event array quantity threshold can include anumber corresponding to an amount of a drug. In cases such as these, theevent array quantity parameter can satisfy the event array quantitythreshold if the event array quantity parameter matches or exceeds theevent array quantity threshold. Alternatively, the event array quantityparameter can satisfy the event array quantity threshold if the eventarray quantity parameter matches or is below the event array quantitythreshold.

The event array quantity thresholds can be determined using variousmethods. For example, in some cases, the event array quantity thresholdsare based at least in part on other individuals, such as individuals anda peer group. For example, in some cases, the event array quantitythresholds can correspond to a median, average, or total amount of adrug dispensed, administered, wasted, and/or returned by one or moreother providers.

As described herein, in addition to determining whether an individualevent array satisfies the event array quantity threshold, theverification system 402 can track the number of event arrays associatedwith a user that do or do not satisfy the event array quantity thresholdor the average quantity of the individual users. This number can becompared with a quantity rule threshold to determine whether theprovider satisfies a quantity rule threshold.

The verification system 402 can determine the quantity rule thresholdbased on individual event array quantity parameters of the event arraysof a set of data. For example, the verification system 402 can determinean average number of event arrays that satisfy the event array quantitythreshold for individual users or the average quantity for individualusers. A deviation or different from the average can then be used as thequantity rule threshold. If the average quantity or average number ofevent arrays for a particular user satisfies the event array quantitythreshold, the verification system 402 can identify the user as anabnormal user or otherwise indicate that the user does not satisfy thequantity rule threshold. For example, for a set of data, if the averagequantity for dispensing morphine is 20 ml and the quantity rulethreshold is ±5 ml, and a user's average dispense of morphine is 30 ml,then the verification system 402 can determine that the user does notsatisfy the quantity rule threshold, and indicate the same to a user ofthe verification system 402.

Location Rules

In some cases, a location associated with an event can be indicative ofdrug misuse. Accordingly, in some cases, one or more location rules canbe applied to an event array to generate an event array parameter. Alocation rule can include instructions to identify various locationscorresponding to the event array, and the result of the application ofthe location rule (for example, the result of performing theinstructions) is the event array parameter.

As a non-limiting example, an event array can include two events: anevent at Location A and associated with Provider A, and an event atLocation B and associated with Provider A. In some cases, a locationrule includes instructions to identify where a location associated withan event. In this example, a first event is associated with Location Aand a second event is associated with Location B. Accordingly, in somecases, the event array location parameter can include an indication ofthe location. In some embodiments, the event array location parameterand event array location threshold can be values.

In some cases, the event array location parameter can be compared to oneor more event array location thresholds to determine whether the eventarray location parameter satisfies the respective event array locationthreshold. In some cases, if the event array location parametersatisfies the event array location threshold, then the events taken by aprovider can be considered normal, usual, acceptable, or the like. Incontrast, in some cases, if the event array location parameter does notsatisfy the event array location threshold, then the events taken by aprovider can be considered abnormal, unusual, unacceptable, or the like.

The event array location threshold can include a location zone, such asa region on a map. In cases such as these, the event array locationparameter can satisfy the event array location threshold if the eventarray location parameter (e.g., the location) is located within thelocation zone. Alternatively, the event array location parameter cansatisfy the event array location threshold if the event array locationparameter matches the event array location threshold.

FIG. 13 illustrates an example hospital floor map indicative of anexample clustering of providers into their respective care areas. Inthis illustration, each point on the map indicates a location of anevent performed by providers. Accordingly, the various clusters 1302,1304, 1306, 1308, 1310, and 1312 illustrate the popular or typicalregions or areas that providers perform the events.

In some cases, an event array location threshold can correspond to aregion on the map, such as a typical region that events are performed.As a non-limiting example, the event array location threshold cancorrespond to region 1340. Accordingly, in this case, if the event arraylocation parameter (e.g., the location of the event) is associated withpoint 1342, then the event array location parameter does not satisfy theevent array location threshold. That is because point 1342 does not fallwithin region 1340. However, if the event array location parameter isassociated with point 1344, then the event array location parameter doessatisfy the event array location threshold. That is because point 1342falls within region 1340.

As described herein, in addition to determining whether an individualevent array satisfies the event array location threshold, theverification system 402 can track the number of event arrays associatedwith a user that do or do not satisfy the event array locationthreshold. This number can be compared with a location rule threshold todetermine whether the provider satisfies a location rule threshold.

The verification system 402 can determine the location rule thresholdbased on individual event array location parameters of the event arraysof a set of data. For example, the verification system 402 can determinean average number of event arrays that satisfy the event array locationthreshold for individual users. A deviation or different from theaverage can then be used as the location rule threshold. If the averagenumber of event arrays for a particular user satisfies the event arraylocation threshold, the verification system 402 can identify the user asan abnormal user or otherwise indicate that the user does not satisfythe location rule threshold. For example, for a set of data, if theaverage number of event arrays that satisfy the location threshold forindividual users is two and the location rule threshold is >3, and auser's number of event arrays that satisfy the location threshold isfive, then the verification system 402 can determine that the user doesnot satisfy the location rule threshold, and indicate the same to a userof the verification system 402.

Witness Rules

In some cases, a pattern of working with another provider to perform awaste event can be indicative of drug misuse. Accordingly, in somecases, one or more witness rules can be applied to an event array togenerate an event array parameter. A witness rule can includeinstructions to identify providers (e.g., performer of the event and thewitness) associated with event array, and the result of the applicationof the witness rule (for example, the result of performing theinstructions) is the event array parameter.

As a non-limiting example, an event array can include two events: anevent associated with provider A and witness B, and an event associatedwith provider A and witness C. In some cases, a witness rule includesinstructions to identify the witness and/or provider associated with adispensing event. Accordingly, in some cases, the event array witnessparameter can include an indication of the witness, the provider, or acombination thereof.

In some cases, a witness rule includes instructions to identify how manytimes a particular group of providers worked together (e.g., asperformer and witness) to waste a drug. Accordingly, in some cases, theevent array witness parameter can include a value.

In some cases, the event array witness parameter can be compared to oneor more event array witness thresholds to determine whether the eventarray witness parameter satisfies the respective event array witnessthreshold. In some cases, if the event array witness parameter satisfiesthe event array threshold, then the events taken by a provider can beconsidered normal, usual, acceptable, or the like. In contrast, in somecases, if the event array witness parameter does not satisfy the eventarray threshold, then the events taken by a provider can be consideredabnormal, unusual, unacceptable, or the like.

The event array witness thresholds can come in many forms and can bebased at least in part on the applied rule. For example, if the witnessrule includes instructions to identify the witness and/or provider pair,then the event array witness threshold can, for example, be a string oftheir names. If the event array witness parameter matches the string,then the event array witness parameter satisfies the event array witnessthreshold. However, if the event array witness parameter does not matchthe string, then the event array witness parameter does not satisfy theevent array witness threshold.

As another example, if the witness rule includes instructions toidentify how many times, over a given period, that a particular pair orgroup of providers worked to waste a drug, then the event array witnessthreshold can be a value. In some cases, if the event array witnessparameter is greater than or equal to that value, then the event arraywitness parameter satisfies the event array witness threshold. In somecases, if the event array witness parameter is less than or equal tothat value, then the event array witness parameter satisfies the eventarray witness threshold.

As described herein, in addition to determining whether an individualevent array satisfies the event array witness threshold, theverification system 402 can track the number of event arrays associatedwith a user that do or do not satisfy the event array witness thresholdor monitor the event arrays associated with the user to identifypatterns of witnesses or groups of users that frequently waste together.This number can be compared with a witness rule threshold to determinewhether the provider satisfies a witness rule threshold.

The verification system 402 can determine the witness rule thresholdbased on individual event array witness parameters of the event arraysof a set of data. For example, the verification system 402 can determinean average number of different witnesses for individual users. Adeviation or different from the average can then be used as the witnessrule threshold. If the average number of event arrays for a particularuser satisfies the event array witness threshold, the verificationsystem 402 can identify the user as an abnormal user or otherwiseindicate that the user does not satisfy the witness rule threshold. Forexample, for a set of data, if the average number of witnesses forindividual users is twelve and the witness rule threshold is ±4, and auser's number of witnesses is two, then the verification system 402 candetermine that the user does not satisfy the witness rule threshold, andindicate the same to a user of the verification system 402.

Behavior Rules

In some cases, a changing behavioral pattern can be indicative of drugmisuse. Accordingly, in some cases, one or more behavioral rules can beapplied to an event array to generate an event array behavior parameter.A behavioral rule can include instructions to identify a provider'sbehavior over a period time, and the application of the behavioral rule(for example, the result of performing the instructions) can result inthe event array behavior parameter.

As a non-limiting example, a behavioral rule can include instructions toidentify, based on a plurality of events or event arrays, patterns ofbehavior of a particular user and determine whether the patterns havechanged. The patterns can include, but are not limited to, the amountthat the provider generally dispenses, wastes, administers, or returns,the typical location at which the provider typically dispenses,administers, wastes, etc., who the provider typically works with (forexample, as a witness to waste events), the typical sequence of eventsperformed by the provider, the amount of time between events, etc. Insome cases, these determinations can be made based on averages ortotals.

To identify the patterns of behavior, the verification system 402 cananalyze a set of data that includes multiple event arrays associatedwith a user over time. From the set of data, the verification system 402can determine a variety of event array parameters, such as, but notlimited to, average time between events, identification of witnesses,average amount dispensed for different drugs, typical sequence ofevents, typical locations for dispensing, administering, etc.

In addition, the verification system 402 can determine the various eventarray parameters for different portions of the set of data. For example,the verification system 402 can break up the set of data into subsets ofdata, determine the event array parameters for the subsets of data, andcompare the event array parameters between the subsets of data or withthe event array parameters of the set of data.

In some cases, the verification system 402 can generate the subsets ofdata based on time. For example, if the verification system 402 breaksup the set of data in to four groups, the first group can be theearliest in time event arrays. The second group can be the secondearliest in time event arrays, the third group can be the third earliestin time event arrays, and the fourth group can include the last in timeevent arrays.

The event array parameter of the different groups can be compared toidentify deviations between the groups. For example, the event arrayparameters for the set of data as a whole can be used as an average orthe event array parameters of the different groups of the set of datacan be used to determine averages for the event array parameters. Theevent array parameter thresholds can be determined as a deviation ordifference from the averages. As such, the event array parameterthresholds can be compared with the event array parameters of thedifferent groups to identify groups that do not satisfy the event arrayparameter thresholds.

In some cases, the system can determine the event array parameters forthe set of data and then compared the determined event array parameterswith future sets of data associated with a user. For example, theverification system 402 can determine the average time between eventsfor a user based on weeks of data. The average time between events canthen be compared with the amount of time between events of new eventarrays to identify changes, etc.

Rules Scores and Overall Scores

In some cases, the verification system 402 can generate a scoreassociated with whether an event array parameter satisfies an eventarray parameter threshold. For example, the score can be indicative of anumber of times (for example, a number of events) in which a particularprovider is associated with an event array parameter that does notsatisfy the event array parameter threshold. For example, the score canbe the total of number of times that the event array parameter that doesnot satisfy the event array parameter threshold. In some cases, thetotal of number of times that the event array parameter that does notsatisfy the event array parameter threshold can be plotted relative toothers, and outlier scores can be created in such a way that they areexponentially distributed, and are then standardized by the mean of theoutlier scores. In some cases, the score value (for example, 5) canindicate that, over a specific time-period (configurable by a user), theassociated provider has 5 times more, 5 times less, 5 more, 5 less, etc.that a normal provider. In some cases, the normal provider can bedetermined using similar rules for all or s subset of providers,exponentially distributing the data, and are then standardizing by themean of the outlier scores.

In some cases, applying the rules to the event arrays can allow thesystem to identify behavior patterns, trends, or other anomalous usageinformation to identify particular providers suspected as a high riskdrug diversion. Moreover, in some cases, the system can be configured todetermine or present actionable recommendations, which can further helpto minimize risk of future diversions.

In some implementations, the verification system 402 can determine anoverall score, which can be indicative of a particular provider'sdiversion risk. For example, in some cases, the overall score can be afunction of one or more of the scores associated with the applied rules,as described above. For example, the overall score can be an average,median, total, etc. of one or more scores. Alternatively, the system canweight one or more of the scores differently from other scores, and theoverall score can be determined based off of the weighted scores. Insome cases, the overall score can provide a holistic overview of asingle provider/nurse across multiple metrics, and providers/nurses canbe easily compared with one-another. Furthermore, by utilizing multiplemetrics, the system can highlight the problem areas of eachprovider/nurse.

FIGS. 14A-14B illustrate example graphical user interfaces (GUIs)displaying various metrics associated with a risk of drug diversion,according to some embodiments. FIG. 14A includes a ranked list ofproviders/nurse 1460, ordered from most unusual/risky to least, based onthe overall score of each provider. As illustrated from FIG. 14A, thepresentation of the overall score 1470 for each provider can allow aviewer to quickly identify a risk of diversion associated with thelisted providers.

In some cases, each overall score 1470 can indicate a difference ascompared to normal behavior (described in more detail below). Forexample, in FIG. 14A, Delia Frances 1402 is listed with a score of 9.1.In some cases, the overall score of 9.1 can indicate that, over aspecific time-period (configurable by a user), Delia Frances is 9.1 moretimes more likely to be a drug diverter than her peers. In other words,the overall score can indicate who, if anyone, is at risk of drugdiversion.

FIG. 14B illustrates a detailed risk score analysis 1480 of ScottMatthews. As illustrated, several of the metrics used to generateScott's overall score 1404 of 5.0 are shown. For example, Scott has a“dispense patterns” score (1412) of 6.7. In some cases, the dispensepatterns score can correspond to the dispensing rules, as describedabove. Further, Scott has a “variance trends” score (1414) of 3.6. Insome cases, the variance trends score can correspond to the omissionrules, as described above. Further, Scott has a “med trends” score(1416) of 1.7. In some cases, the med trends score can correspond to thesequence rules, as described above. Further, Scott has an “action times”score (1418) of 8.1. In some cases, the action times score cancorrespond to the timing rules, as described above.

While Scott's overall score (5.0) provides a holistic overview ofScott's drug diversion risk to the medical facility, the other metricsexplain or define Scott's particular problem areas. Specifically, inthis example, Scott has a very high Dispense Patterns and Action Timesscore. In some cases, as illustrated in FIG. 14B, the system isconfigured to display natural language in addition to each metric.Accordingly, medical facility personnel can see the metric, and can alsounderstand the reasoning and why a provider/nurse is consideredrisky/unusual.

Terminology

Conditional language, such as, among others, “can,” “could,” “might,” or“may,” unless specifically stated otherwise, or otherwise understoodwithin the context as used, is generally intended to convey that certainembodiments include, while other embodiments do not include, certainfeatures, elements, and/or steps. Thus, such conditional language is notgenerally intended to imply that features, elements and/or steps are inany way required for one or more embodiments or that one or moreembodiments necessarily include logic for deciding, with or without userinput or prompting, whether these features, elements and/or steps areincluded or are to be performed in any particular embodiment.

Unless the context clearly requires otherwise, throughout thedescription and the claims, the words “comprise,” “comprising,” and thelike are to be construed in an inclusive sense, as opposed to anexclusive or exhaustive sense; that is to say, in the sense of“including, but not limited to.” As used herein, the terms “connected,”“coupled,” or any variant thereof means any connection or coupling,either direct or indirect, between two or more elements; the coupling orconnection between the elements can be physical, logical, or acombination thereof. Additionally, the words “herein,” “above,” “below,”and words of similar import, when used in this application, refer tothis application as a whole and not to any particular portions of thisapplication. Where the context permits, words in the above DetailedDescription using the singular or plural number may also include theplural or singular number respectively. The word “or” in reference to alist of two or more items, covers all of the following interpretationsof the word: any one of the items in the list, all of the items in thelist, and any combination of the items in the list. Likewise the term“and/or” in reference to a list of two or more items, covers all of thefollowing interpretations of the word: any one of the items in the list,all of the items in the list, and any combination of the items in thelist.

Depending on the embodiment, certain operations, acts, events, orfunctions of any of the algorithms described herein can be performed ina different sequence, can be added, merged, or left out altogether(e.g., not all are necessary for the practice of the algorithms).Moreover, in certain embodiments, operations, acts, functions, or eventscan be performed concurrently, e.g., through multi-threaded processing,interrupt processing, or multiple processors or processor cores or onother parallel architectures, rather than sequentially.

Systems and modules described herein may comprise software, firmware,hardware, or any combination(s) of software, firmware, or hardwaresuitable for the purposes described herein. Software and other modulesmay reside and execute on servers, workstations, personal computers,computerized tablets, PDAs, and other computing devices suitable for thepurposes described herein. Software and other modules may be accessiblevia local memory, via a network, via a browser, or via other meanssuitable for the purposes described herein. Data structures describedherein may comprise computer files, variables, programming arrays,programming structures, or any electronic information storage schemes ormethods, or any combinations thereof, suitable for the purposesdescribed herein. User interface elements described herein may compriseelements from graphical user interfaces, interactive voice response,command line interfaces, and other suitable interfaces.

Further, the processing of the various components of the illustratedsystems can be distributed across multiple machines, networks, and othercomputing resources. In addition, two or more components of a system canbe combined into fewer components. Various components of the illustratedsystems can be implemented in one or more virtual machines, rather thanin dedicated computer hardware systems and/or computing devices.Likewise, the data storage devices shown can represent physical and/orlogical data storage, including, for example, storage area networks orother distributed storage systems. Moreover, in some embodiments theconnections between the components shown represent possible paths ofdata flow, rather than actual connections between hardware. While someexamples of possible connections are shown, any of the subset of thecomponents shown can communicate with any other subset of components invarious implementations.

Embodiments are also described above with reference to flow chartillustrations and/or block diagrams of methods, apparatus (systems) andcomputer program products. Each block of the flow chart illustrationsand/or block diagrams, and combinations of blocks in the flow chartillustrations and/or block diagrams, may be implemented by computerprogram instructions. Such instructions may be provided to a processorof a general purpose computer, special purpose computer,specially-equipped computer (e.g., comprising a high-performancedatabase server, a graphics subsystem, etc.) or other programmable dataprocessing apparatus to produce a machine, such that the instructions,which execute via the processor(s) of the computer or other programmabledata processing apparatus, create means for implementing the actsspecified in the flow chart and/or block diagram block or blocks.

These computer program instructions may also be stored in anon-transitory computer-readable memory that can direct a computer orother programmable data processing apparatus to operate in a particularmanner, such that the instructions stored in the computer-readablememory produce an article of manufacture including instruction meanswhich implement the acts specified in the flow chart and/or blockdiagram block or blocks. The computer program instructions may also beloaded onto a computing device or other programmable data processingapparatus to cause a series of operations to be performed on thecomputing device or other programmable apparatus to produce a computerimplemented process such that the instructions which execute on thecomputer or other programmable apparatus provide steps for implementingthe acts specified in the flow chart and/or block diagram block orblocks.

Any patents and applications and other references noted above, includingany that may be listed in accompanying filing papers, are incorporatedherein by reference. Aspects of the invention can be modified, ifnecessary, to employ the systems, functions, and concepts of the variousreferences described above to provide yet further implementations of theinvention.

These and other changes can be made to the invention in light of theabove Detailed Description. While the above description describescertain examples of the invention, and describes the best modecontemplated, no matter how detailed the above appears in text, theinvention can be practiced in many ways. Details of the system may varyconsiderably in its specific implementation, while still beingencompassed by the invention disclosed herein. As noted above,particular terminology used when describing certain features or aspectsof the invention should not be taken to imply that the terminology isbeing redefined herein to be restricted to any specific characteristics,features, or aspects of the invention with which that terminology isassociated. In general, the terms used in the following claims shouldnot be construed to limit the invention to the specific examplesdisclosed in the specification, unless the above Detailed Descriptionsection explicitly defines such terms. Accordingly, the actual scope ofthe invention encompasses not only the disclosed examples, but also allequivalent ways of practicing or implementing the invention under theclaims.

To reduce the number of claims, certain aspects of the invention arepresented below in certain claim forms, but the applicant contemplatesthe various aspects of the invention in any number of claim forms. Anyclaims intended to be treated under 35 U.S.C. § 112(f) will begin withthe words “means for”, but use of the term “for” in any other context isnot intended to invoke treatment under 35 U.S.C. § 112(f). Accordingly,the applicant reserves the right to pursue additional claims afterfiling this application, in either this application or in a continuingapplication.

1. A method for determining event discrepancies between disparatesystems, the method comprising: querying a plurality of systems for datarelated to one or more events; receiving a first set of events from afirst system of the plurality of systems, each event of the first set ofevents comprising a plurality of event parameters; receiving a secondset of events from a second system of the plurality of systems, eachevent of the second set of events comprising a plurality of eventparameters, wherein the first set of events are different from and notcompatible with the second set of events; normalizing the first set ofevents to generate a normalized first set of events and normalizing thesecond set of events to generate a normalized second set of events;concurrently generating a plurality of event arrays, at least one eventarray of the plurality of event arrays comprising a plurality ofnormalized events that occurred within a particular window of time andthat include at least one matching event parameter, the plurality ofevents including at least a first normalized event corresponding to afirst event of the first set of events and a second normalized eventcorresponding to a second event of the second set of events; applying atleast one rule to the at least one event array to generate at least oneevent array parameter; comparing the at least one event array parameterto a corresponding event array parameter threshold to determine that theat least one event array parameter satisfies the event array parameterthreshold; and based on said comparing, indicating to a user that the atleast one array parameter satisfies the event array parameter threshold.2. The method of claim 1, wherein event parameters comprise one or moreof a drug identifier, an action identifier identifying an action, atiming identifier, a patient identifier, a provider identifier or alocation identifier.
 3. The method of claim 2, wherein the drugidentifier comprises an indication of at least one of a drug name, adrug quantity, a drug concentration, or a drug package size.
 4. Themethod of claim 2, wherein the action identifier comprises an indicationof an action associated with a drug, wherein the action comprises atleast one of a drug dispensing action, a drug administration action, adrug waste action, or a drug return action.
 5. The method of claim 2,wherein the timing identifier comprises an indication of a date and/ortime associated with the action.
 6. The method of claim 2, wherein thepatient identifier comprises an indication of a patient associated withthe action, the indication of the patient comprising at least one of apatient name, birth date, height, weight, or social security number. 7.The method of claim 2, wherein the location identifier comprises anindication of a location associated with the action.
 8. The method ofclaim 1, wherein the first set of events are generated by the firstsystem, and the second set of events are generated by the second system.9. The method of claim 1, wherein the first system and/or the secondsystem comprise a verification system, a dispensing system, a patientinformation system, an administration system, or a billing system. 10.The method of claim 1, wherein said querying the plurality of systemsoccurs responsive to a user command.
 11. The method of claim 1, whereinsaid querying the plurality of systems occurs at predetermined intervalsof time.
 12. The method of claim 1, wherein said querying the pluralityof systems comprises querying the first system at a first time andquerying the second system at a second time that is different from thefirst time.
 13. The method of claim 1, wherein the normalized first setof events and the normalized second set of events comprise a commonformat.
 14. The method of claim 1, wherein the normalized first set ofevents and the normalized second set of events comprise same eventparameter fields.
 15. The method of claim 8, wherein said normalizingthe first set of events and the second set of events comprises:identifying event parameters from the plurality of event parameters ofthe first set of events and the second set of events for inclusion inthe normalized first set of events and the normalized second set ofevents; and discarding event parameters of the first set of events andthe second set of events that are not identified for inclusion in thenormalized first set of events and the normalized second set of events.16. The method of claim 8, wherein said normalizing the first set ofevents and the second set of events comprises: comparing eventparameters of the first set of events and the second set of events toidentify an event of the first set of events that matches an event ofthe second set of events; and discarding the event of the first set ofevents that matches the event of the second set of events.
 17. Themethod of claim 1, wherein the at least one rule comprises a sequencedetermination rule to identify a sequence of events in the at least oneevent array, wherein the at least one event array parameter comprisesthe sequence of events.
 18. The method of claim 1, wherein the at leastone rule comprises a dispensing determination rule to identify whetheran event of a first type follows an event of a second type in the atleast one event array. 19-38. (canceled)
 39. A system for determiningevent discrepancies between disparate systems, the system comprising oneor more processors configured to: query a plurality of systems for datarelated to one or more events; receive a first set of events from afirst system of the plurality of systems, each event of the first set ofevents comprising a plurality of event parameters; receive a second setof events from a second system of the plurality of systems, each eventof the second set of events comprising a plurality of event parameters,wherein the first set of events are different from and not compatiblewith the second set of events; normalize the first set of events togenerate a normalized first set of events and normalizing the second setof events to generate a normalized second set of events; concurrentlygenerate a plurality of event arrays, at least one event array of theplurality of event arrays comprising a plurality of normalized eventsthat occurred within a particular window of time and that include atleast one matching event parameter, the plurality of events including atleast a first normalized event corresponding to a first event of thefirst set of events and a second normalized event corresponding to asecond event of the second set of events; apply at least one rule to theat least one event array to generate at least one event array parameter;compare the at least one event array parameter to a corresponding eventarray parameter threshold to determine that the at least one event arrayparameter satisfies the event array parameter threshold; and based onthe comparison, cause an indication to a user that the at least onearray parameter satisfies the event array parameter threshold.
 40. Anon-transitory computer-readable storage medium storingcomputer-executable instructions that when executed by one or moreprocessors cause the one or more processors to: query a plurality ofsystems for data related to one or more events; receive a first set ofevents from a first system of the plurality of systems, each event ofthe first set of events comprising a plurality of event parameters;receive a second set of events from a second system of the plurality ofsystems, each event of the second set of events comprising a pluralityof event parameters, wherein the first set of events are different fromand not compatible with the second set of events; normalize the firstset of events to generate a normalized first set of events andnormalizing the second set of events to generate a normalized second setof events; concurrently generate a plurality of event arrays, at leastone event array of the plurality of event arrays comprising a pluralityof normalized events that occurred within a particular window of timeand that include at least one matching event parameter, the plurality ofevents including at least a first normalized event corresponding to afirst event of the first set of events and a second normalized eventcorresponding to a second event of the second set of events; apply atleast one rule to the at least one event array to generate at least oneevent array parameter; compare the at least one event array parameter toa corresponding event array parameter threshold to determine that the atleast one event array parameter satisfies the event array parameterthreshold; and based on the comparison, cause an indication to a userthat the at least one array parameter satisfies the event arrayparameter threshold.